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ABSTRACT 


This  study  evaluates  the  accuracy  of  established  reliability  prediction  tech¬ 
niques  applied  to  a  cross  section  of  ground  electronic  equipment.  Pre-d^ign 
and  stress  analysis  prediction  methods  were  evaluated,  including  those  o^  the 
revised  RADC  Reliability  Notebook,  Volume  II,  dated  Septen&er  1967. 

The  study  objectives  were  to  measure  and  improve  the  accuracy  of  existing  pre¬ 
design  and  stress  analysis  reliability  prediction  techniques,  and  to  apply 
the  results  to  the  development  of  a  recommended  reliability  demonstration 
approach. 

Prediction  accuracy  was  determined  by  obtaining  measured  reliability  data  from 
field  experience  for  comparison  with  new  reliability  predictions  prepared  for 
the  same  equipment  in  accordance  with  the  established  prediction  techniques. 
Special  studies  and  correlation  analyses  were  performed  to  identify  and  esta¬ 
blish  relationships  significant  to  the  identification  and  correction  of  fac¬ 
tors  responsible  for  inaccuracies  in  the  prediction  techniques.  Program- 
related  factors  were  evaluated  and  a  quantitative  program  rating  system  was 
developed  for  correlation  with  prediction  accuracy  values. 

Sensitivity  studies  were  performed  to  determine  the  degree  to  which  stress, 
temperature,  and  quality  grade  factors  influence  the  stress  analysis  reli¬ 
ability  predictions  for  a  variety  of  equipment  types.  The  behavior  of  equip¬ 
ment  failure  rates  during  extended  field  operations  was  determined  and  eval¬ 
uated  for  a  variety  of  equipment  functional  types.  Design  approach  categories, 
and  functional  categories,  were  established  and  applied  to  identify  equipment 
groups  having  common  characteristics  resulting  in  distinct  ranges  of  predic¬ 
tion  accuracy. 

Reliability  demonstration  techniques  for  ground  electronics  equipment  were  re¬ 
viewed  and  a  recommended  demonstration  approach  was  established. 


EVALLRTIOK 


1.  The  objectives  of  inis  study  were  to  measure  and  improve  uie 
accuracy  of  existing  ore -design  end  stress  analysis  prediction 
te-c.»niquec.  on  ground  electronic  equipment .  ?ne  study  also  cal  leu 
for  a  review  of  reliability  demonstration  tec  uniques  an-  t.je 
•e/eloprject  of  a  recommenced  demonstration  approacn,  vnic.:  utilizes 
a  ricr  ^reoietlon,  for  ground  equipment* 

*-•  Tdcse  objectives  were  satisfactorily  acco/rpiisne ..  Modifiers, 
v  lien  e.*iia.-jce  t  ie  accuracy  and  precision  of  present  prediction 
.net nods,  were  ueveiopen,  ar,-  a  pressed  deiay’-stra-ion  approacn.  using 
t:e  reliability  prediction  as  &  wise line,  is  presented.  Section  VI 
is  of  special  interest  Decause  it  includes  a  comprehensive  analysis  of 
integrated  circuit  failure  rate  data  and  an  evaluation  of  tne  relative 
accijracy  of  tne  RAdC  and  EIA  integrated  circuit  prediction  models . 

A itno'jgn  toe  prediction  modifiers  developed  in  tnis  study  improve 
tne  accuracy  oi  existing  teenniques,  tne  results  of  tnis  study  also 
clearly  Indicate  tne  need  for  major  improvements  in  tiie  prediction 
process-  For  example,  it  is  evident  tnat  separate  techniques  are 
required  for  digital  and  analog  equip;aents,  and  tnat  consideration 
saould  i>e  given  to  items  sue a  as  multilayer  printed  circuit  boards 
v.dcij  are  not  included  in  present  predictions.  Future  RA.EC  studies 
in  reliability  prediction  will  be  directed  towards  tne  e Urination  of 
tnese  deficiencies  and  otners  vnicu  were  discovered  in  tnis  effort. 
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SECTION  I 


INTRODUCTION 

This  study  had  the  primary  objective  of  evaluating  and  improving  the  accuracy 
of  pre-design  and  stress  analysis  reliability  prediction  techniques  applied 
to  a  variety  of  functional  types  of  ground  electronic  equipment. 

Emphasis  was  placed  upon  the  recently  revised  (September  1967)  stress  analysis 
prediction  technique  of  the  RADC  Reliability  Notebook,  Volume  II,  RADC  TR-67- 
108. 

A  secondary  objective  was  to  develop  a  reliability  demonstration  approach. 

This  involved  the  investigation  of  demonstration  methods,  reliability  assur¬ 
ance  techniques,  and  program  characteristics  which  can  serve  to  diminish  or 
eliminate  the  need  for  formal  demonstration  testing.  Also  Included  were 
the  investigation  of  reliability  program  attributes,  reliability  program 
ratings,  and  other  program  variables  which  are  known  to  contribute  to  the 
control  and  elimination  of  design  weaknesses  and  part  failures,  thereby 
increasing  confidence  in  the  achievement  of  reliability  objectives. 

The  technical  approach  employed  and  the  tasks  performed  to  achieve  these 
objectives  are  described  in  Section  II,  "Surosary,"  which  follows. 
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SECTION  II 
SUMMARY 


This  section  supplements  the  Information  presented  In  the  “Abstract11  In  order 
to  provide  a  more  detailed  description  of  the  technical  approach  and  the  tasks 
performed  during  the  study.  The  principal  tasks  Included  the  following: 

o  Collection  of  observed  reliability  data,  and  equipment  baseline  configura¬ 
tion  data  for  the  preparation  of  reliability  predictions. 

o  Preparation  and  evaluation  of  stress  analysis  and  pre-design  reliability 
prediction  techniques. 

o  Improvement  of  reliability  prediction  techniques. 

o  Collection  and  analysis  of  Integrated  circuit  design,  operation,  and 
failure  data. 

o  Performance  of  special  studies  to  Investigate  significant  areas  pertinent 
to  the  evaluation  and  Improvement  of  stress  analysis  and  pre-design 
reliability  prediction  techniques. 

o  Evaluation  of  reliability  test  and  demonstration  approaches  for  ground 
electronics  equipment. 

The  technical  approach  emphasized  the  comparative  analysis  of  predicted  and 
observed  failure  rates,  using  a  variety  of  fixed  ground  electronics  equip¬ 
ment  representing  a  cross  section  of  the  types  currently  In  the  Air  Force 
Inventory.  Statistical  analyses  were  performed  at  all  Indenture  levels  of 
equipment,  from  the  system  through  the  part  level,  to  evaluate  relationships 
between  prediction  accuracy  and  such  factors  as: 

o  Strength  of  reliability  program 

o  Design  approach 

o  Function 

o  Prediction  method 

o  Stress,  temperature,  and  part  grade 

o  Parts  technology 

o  Design  complexity 

This  report  presents  the  results  of  these  tasks  and  analyses  In  the  sections 
which  follow.  Each  analytical  section  Is  introduced  by  a  subsection  pre¬ 
senting  the  objectives  of  the  analysis.  Conclusions  are  presented  in  the 
summary  portion  In  each  section. 
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SECTION  III 


* 


DATA  USED  IN  STUDY 

1 .  INTRODUCTION 

This  section  presents  a  tabulation  of  the  data  employed  in  performing 
this  study.  The  data  tabulated  under  III  2  "Candidate  Equipments"  were 
used  throughout  the  study  except  for  Special  Studies  No.  1  and  No.  5  of 
Section  VIII.  The  data  utilized  in  Section  VIII,  Special  Studies  No.  1 
and  No.  5  are  tabulated  under  II 1-3  "Data  for  Special  Studies  No.-  1  and 
No.  5." 

2.  CANDIDATE  EQUIPMENTS 

The  data  from  thirty-seven  pieces  of  electronic  equipment,  operating  in  a 
fixed-ground  environment,  were  selected  for  utilization  in  this  study 
(except  for  Section  VIII,  Special  Study  No.  1  and  Special  Study  No.  5). 
The  selected  equipments  encompass  the  full  spectrum  of  ground  electronic 
equipment  types  (e.g.,  radars,  receivers,  displays,  computers,  etc.). 

The  design  dates  of  the  equipment  vary  from  1959  through  1967  and  include 
all  types  of  electronic  components  (e.g.,  tubes,  transistors,  and  inte¬ 
grated  circuits).  The  selection  of  these  equipments  was  made  considering 
the  following  criteria,  but  not  necessarily  in  the  order  of  listing: 

o  Availability  of  equipment  parts  lists 

o  Availability  of  contractor  prediction  on  equipment 

o  Availability  of  operating  failure  data  on  the  mature  equipment 

o  Established  failure  reporting  system  (e.g.:  AF66-I,  contractor 
reporting,  etc.) 

o  Validity  of  date  for  desired  analysis  techniques. 

In  Table  I  “Ground  Electronics  Study  Equipment  Parts  breakdown,"  the  equip¬ 
ments  are  listed  by  code  numbers,  total  parts,  and  percentage  of  total  parts 
for  each  part  tyoe.  This  table  is  very  useful  in  reflecting  the  distribution 
(i.e.,  parts  mix)  of  parts  in  ground  electronic  equipments. 

3.  DATA  FOR  SPECIAL  STUDIES  NO.  1  AND  NO.  5 


Suninary 


The  data  tabulated  in  this  section  is  used  in  Special  Studies  No.  1  and 
No.  5.  The  data  are  field  data  on  ground  electronic  equipment  taken  over 
a  calendar  period  of  seven  years,  ending  in  1965.  The  equipments  are 
broadly  categorized  as  either  transmitter  type,  receiver  type,  computer 
type,  or  display  type  equipment.  The  equipment  was  maintained  by  contrac¬ 
tor  personnel  and  USAF  agencies  and  exhibited  a  constant  duty  cycle. 
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TABLE  I  GROUND  ELECTRONICS  STUDY  EQUIPMENT  PARTS  BREAKDOWN 
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TABLE  I  GROUND  ELECTRONICS  STUDY  EQUIPMENT  PARTS  BREAKDOWN  (CONTINUED) 


TABLE  I  GROUND  ELECTRONICS  STUDY  EQUIPMENT  PARTS  BREAKDOWN  (CONCLUDED) 
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Tne  failure  definition  used  in  this  study  includes  all  failures  that 
caused  a  downtime  or  interruption  to  the  operational  performance  of 
the  equipment  excluding  externally  caused  failures,  such  as  power 
outages.  Therefore,  removal  and  replacesaent  actions  were  not  associ¬ 
ated  with  all  failure  events.  All  equipment  was  in  the  steadjy- 
state  operating  porfon  of  the  equipaent  life  cycle  {i.e.,  all 
equipment  had  accumulated  apprcxinately  8-10,000  hours  of  operating 
time).  Thus,  the  data  excludes  the  debugging  and  installation  phases 
of  tne  equipment  life  cycle.  A  sifasary  of  the  accumulated  unit 
operating  hours  included  within  each  equipment  type,  is  given  be'ow. 


o.  Qualitative  Discussion  of  the  Field  Data 

All  the  data  analyzed  in  this  section  of  the  report  were  obtained 
frr®  summarized  field  reliability  reports  on  the  particular  equip¬ 
ments.  The  reports  discussed  the  causes  of  some  cf  the  failures  that 
occurred,  including  failures  due  to  the  following  causes: 

(1)  Design  Inadequacies 

Even  during  this  period  of  maturity  in  the  life  cycle  a  number 
of  design  inadequacies  became  evident  during  the  operation  of 
the  equipment.  After  recommended  corrective  action  was  taken, 
the  equipment  failure  rate  shewed  a  decrease,  in  some  cases, 
which  might  be  attributed  to  the  action  taken. 

(2)  Parts  Deficiencies 

Deficiencies  in  some  parts  characteristics  are  indicated  to  have 
contributed  to  equipment  failure  rates  in  several  instances. 
However,  the  extent  of  the  contribution  could  not  be  quantita¬ 
tively  established,  and  could  not  always  be  clearly  distinguished 
from  the  category  of  design  inadequacies.  Examples  of  the  types 
of  parts  deficiencies  which  are  known  to  be  present  in  all 
fielded  systems  include:  quality  control  deficiencies,  defici¬ 
encies  due  to  shipping  and  handling,  and  deficiencies  caused  by 
stresses  incurred  during  installation  and  checkout  activities. 
This  category  also  includes  part  failures  due  to  configuration 
control  problems. 


Type  Equipment 

Operating  Hours 

% 

c  Transmitter 

527,060 

1 

o  Receiver 

1 ,792,630 

! 

/■ 

o  Display 

543,544 

* 

j 

o  Cocputer 

887,003 

| 
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i 
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(3)  falntenance  and  Operational  Policies  and  Practices 

Like  all  other  systems,  these  equipments  were  subject  to  the 
influence  of  changing  maintenance  and  operational  policies  and 
practices.  These  changes  result  in  both  increased  and  reduced 
downtime  and  part  replacement.  To  the  extent  that  changes  are 
made,  equipment  failure  rates  are  indicated  to  be  a  function  of 
these  changes. 

(4)  Part  Wearout  Failures 

Certain  parts,  such  as  tubes ,  relays,  motors  and  certain  moving 
mechanical  components,  have  known  wearout  mechanisms.  When  these 
parts  wear  out,  they  are  reported  as  field  failures  and  are 
included  in  the  equipment  failure  rate. 

(5)  Catastrophic  Failures 

The  above  four  categories  described  failures,  both  catastrophic 
and  non-catas trophic  which  are  traceable  to  the  effects  of  time 
and/or  causes.  This  category  differs  from  the  above  in  that  the 
failures  are  random  in  time  and  may  be  considered  random  in  cause 
These  failures  are  not  attributed  to  a  design  deficiency,  part 
deficiency  or  maintenance  or  operating  policy. 

It  is  recognized  that  all  of  these  failure  categories  are  represented 
in  all  field  data  covering  extended  time  periods.  However,  current 
prediction  techniques  are  primarily  based  on  catastrophic  failures, 
although  wearout  failures  are  recognized  in  the  RADC  stress  analysis 
method.  The  other  failure  classes  are  not  represented  in  the  predic¬ 
tion  techniques.  However,  for  some  equipments  during  selected  periods 
failures  in  these  other  classes  can  represent  the  bulk  of  the  total 
equipment  failures. 

As  discussed,  the  prediction  techniques  generally  yield  higher  than 
observed  failure  rates.  This  inaccuracy  is  accentuated  for  those 
equipments  whose  failure  rate  is  decreasing  with  tine.  The  predic¬ 
tions  would  appear  eyen  more  inaccurate  if  the  failures  due  to  causes 
not  included  in  the  prediction  process  could  be  deleted  from  the 
observed  data. 

The  contribution  to  prediction  inaccuracy  of  failure  classes  net 
represented  in  the  prediction  techniques  could  be  assessed  if  it  were 
possible  to  determine  the  extent  to  which  these  failures  contribute 
to  the  total  equipment  failure  rate.  To  accomplish  this  requires  more 
information  and  analysis  than  is  currently  available  from  any  field 
reporting  system.  Therefore,  all  of  the  reported  failures  have  been 
accepted  as  representative  reliability  statistics,  applicable  to  the 
purposes  of  this  study. 
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c.  Equipment  Analyzed 

Tables  II  through  V  list  the  equipment  which  forms  the  baseline  for 
this  study.  Each  table  lists  the  number  of  equipments  in  each  cate¬ 
gory,  total  operating  hours,  total  equipment  hours,  and  the  total 
number  of  failures.  It  will  be  noted  from  the  tables  that  all  equip¬ 
ments  were  placed  into  one  of  four  broad  functional  categories,  based 
upon  their  functional  application.  Within  each  category,  equipments 
were  divided  into  subgroups  having  identical  characteristics. 

The  transmitter  type  equipment  included  28  individual  equipments 
(e.g.,  transmitters)  grouped  into  four  subgroups  (T-l  through  T-4). 

In  each  subgroup,  the  individual  equipments  each  operated  for  a  period 
of  from  17,640  hours  to  19,310  hours. 

The  receiver-type  equipment  comprised  92  distinct  operational 
equipments  (e.g.,  receivers,  local  oscillators,  intermediate  frequency 
equipment,  active  filters,  etc.).  These  were  grouped  into  eight 
subgroups  (R-l  through  R-8),  in  each  of  which  the  individual  equipments 
operated  for  an  average  of  from  19,340  hours  to  19,530  hours. 

The  computer  equipment  consists  of  two  complete  computer  equipment 
groups  including  core  memory  (C-2  and  C-7),  a  total  of  four  computers, 
an  individual  central  processor  (C-ll),  its  associated  memory  core 
(C-12)  and  a  set  of  magnetic  tape  units  (C-15).  Additional  computer 
type  equipment  consisted  of  computer  interface  units  (C-l,  C-3,  C-6, 
C-8,  C-l 4) ,  a  display  console  controller  ( C-l 3)  and  four  master 
timing  units  (C-4,  C-5,  C-9,  C-IO).  The  failures  cf  the  magnetic 
tape  units  are  classified  as  electrical  and  mechanical  since  these 
two  general  classes  of  failures  are  expected  to  exhibit  different 
failure  rate  versus  time  characteristics. 

The  display  equipment  consists  exclusively  of  display  consoles  (D-l 
thru  D-7)  and  camera  projection  equipment  (D-8,  D-9).  The  latter 
equipment  was  used  to  provide  a  large  screen  dynamic  display  for  one 
of  the  systems  analyzed.  Only  5,994  hours  were  reported  on  this  equip¬ 
ment,  representing  a  small  segment  of *'be' equipment  life.  The 
remaining  34  display  consoles  accumulated  537,550  hours,  an  average  of 
15,800  hours  per  display  unit. 

This  data  is  analyzed  in  detail  in  Special  Studies  No.  1  and  No.  5. 
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TABLE  II 


TRANSMITTER  TYPE  EQUIPMENT 


Equipment 

Quanti  ty 

Total  Operating 
Hours  (Per  Equipment) 

Total  Equipment 

Hours 

Total 

Failures 

T-l 

6 

17,640 

105,840 

2179 

T-2 

10 

19,310 

193,100 

3126 

T-3 

4 

18,750 

75,000 

1470 

T-4 

8 

19,140 

153,120 

2365 

TOTAL 

28 

527 ,060 

9140 

TABLE  III 

RECEIVER  TYPE  EQUIPMENT 


Equipment 

Quantity 

. 

Total  Operating 
Hours  (Per  Equipment) 

Total  Equipment 

Hours 

Total 

Failures 

R-l 

18 

19,530 

351 ,540 

371 

R-2 

16 

19,530 

312,480 

247 

R~3 

13 

19,530 

253,890 

37 

R-4 

14 

19,500 

273,000 

371 

R-5 

11 

19,370 

213,070 

327 

R-6 

10 

19,340 

193,400 

23 

R-7 

6 

19,530 

117,180 

8 

R-8 

4 

19,530 

78,120 

14 

WK 

92 

““1,7927680 

1398 
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TABLE  IV 

COMPUTER  TYPE  EQUIPMENT 


Equipment 

Quanti ty 

Total  Operating 
Hours  (Per  Equipment) 

Total  Equipment 

Hours 

Total 

Failures 

C-l 

14 

19,530 

273,820 

100 

C-2 

2 

19,530 

39,060 

295 

C-3 

2 

19,530 

39,060 

167 

C-4 

2 

19,430 

38,860 

13 

C-5 

3 

19,530 

58,590 

21 

C-6 

11 

19,440 

213,840 

86 

C-7 

2 

19,530 

39,060 

210 

C-8 

2 

19,530 

39,060 

29 

C-9 

2 

19,530 

39,060 

4 

C-10 

2 

19,530 

39,060 

43 

C-ll 

1 

11,158 

11,158 

16 

C-l  2 

1 

11,158 

11,158 

36 

C-l  3 

1 

11,912 

11,912 

9 

C-l  4 

1 

11,389 

11,389 

27 

C-15(E1ec) 

14 

— 

11,158 

97 

C-15(Mech) 

14 

-- 

11,158 

130 

TOTAL 

60 

887,003 

1283 

TABLE  V 


DISPLAY  TYPE  EQUIPMENT 


Equipment 

Quanti ty 

Total  Operating 
Hours  {Per  Equipment) 

— 

Total  Equipment 

Hours 

Total 

Failures 

D-I 

6 

19,480 

116,880 

40 

D-2 

4 

19,530 

78,120 

55 

D-3 

3 

17,360 

52,080 

5 

D-4 

4 

19,530 

78,120 

37 

D-5 

3 

19,530 

58,590 

74 

D-6 

3 

10,850 

32,550 

1 

D-7 

n 

11,110 

122,210 

370 

D-8 

1 

1,830 

1,830 

28 

D-9 

1 

3,164 

3,164 

9 

TOTAL 

36 

543,544 

619 

SECTION  IV 
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IMPROVEMENT  OF  STRESS  ANALYSIS  PREDICTION  TECHNIQUES  5 


1 .  OBJECTIVES 

The  primary  objectives  of  this  section  are  the  assessment  of  the  accuracy 
of  the  different  stress  analysis  prediction  techniques,  and  the  develop¬ 
ment  of  a  more  accurate  prediction  method  for  ground  electronic  equipment. 

2.  SUMMARY 

The  development  of  an  improved  stress  analysis  prediction  technique  is 
predicated  on  the  conclusions  f»tjm  this  investigation.  That  is,  a  defi¬ 
nite  difference  exists  between  digital  and  analog  equipments.  A  modifier 
to  the  RADC  Volume  II  Stress  Analysis  prediction  technique  could  enhance 
the  accuracy  of  the  prediction. 

The  improved  prediction  technique  is  the  application  of  a  modifier  to  the 
RADC,  Volume  II  Reliability  Prediction.  A  separate  modifier  was  developed 
for  both  digital  equipments  and  analog  equipments.  The  modifiers  were 
developed  by  eliminating,  from  the  analog  and  digital  equipments,  those 
equipments  with  tube  components  and  determining  a  least-squares  curve  fit 
to  the  remaining  equipment.  The  least-squares  fit  determined  that  a  linear 
curve  of  the  type 

y  =  a  +  BX,  (1) 

where 

y  s  the  observed  MTBF, 

X  =  the  RADC  Volume  II  Predicted  MTBF,  and 
a,  S,  are  regression  coefficients, 
sufficiently  models  the  data  for  both  types  of  equipments. 

The  regression  coefficients  determined  for  the  digital  equipments  are 
a  =  3.14 
0  =  0.14 

Hence,  to  obtain  an  adjusted  prediction  for  digital  equipments  the  follow¬ 
ing  linear  expression  was  used: 

y  =  3.14  +  0.14X, 


where 
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y  i  the  adjusted  prediction,  and 
X  =  the  original  RADC,  Volume  II  prediction. 


Table  VI  “Adjusted  Prediction  Accuracy  Ratios  vs  RADC,  Volume  II  Pre¬ 
diction  Accuracy  Ratios-Digltal  Equipments”  compares  the  accuracy  of  the 
two  prediction  techniques  for  each  equipment  utilized  In  establishing  the 
modifier. 


TABLE  VI 

ADJUSTED  PREDICTION  ACCURACY  RATIOS  VS  RADC  VOLIWE  II 
PREDICTION  ACCURACY  RATIOS  -  DIGITAL  EQUIPMENTS 


Equipment 
Code  No. 

~ rAdC 

Ratio 

Adjusted 

Ratio 

1 

1.7 

0.4 

5 

6-1 

3.8 

6 

3.8 

i.i 

7 

7.0 

3.7 

8 

3.8 

1.7 

10 

2.1 

1.0 

29 

1.3 

0.8 

32 

11.4 

0.9 

37 

7.1 

1.0 

The  mean  for  the  RADC  Volume  II  prediction  accuracy  ratios  is  4.9  and  the 
mean  for  the  adjusted  prediction  accuracy  ratios  is  1.6  (i.e.,  a  reduction 
of  more  than  200  percent  in  the  inaccuracy).  The  standard  deviation  for 
the  RADC  Volume  II  prediction  accuracy  ratios  is  3.54  and  the  standard 
deviation  for  the  adjusted  prediction  accuracy  ratios  is  1.36  (i.e.,  a 
reduction  of  160  percent  In  the  imprecision).  Hence,  the  adjusted  predic¬ 
tion  technique  will  not  only  result  in  more  accurate  predictions,  but  will 
also  result  in  more  precise  predictions. 


The  analog  equipments  included  eleven  equipments.  The  least-square  curve 
fit  produced  the  following  correlation  coefficients  for  these  equipment;,. 


The  adjusted  predictions  were  obtained  from 
y  =  1.2  +  0.42  X, 

where 

y  =  the  adjusted  MTBF,  and 

X  =  the  original  RADC  Predicted  MTBF. 

Table  VII,  "Adjusted  Prediction  Accuracy  Ratios  vs  RADC  Volume  II  Predic¬ 
tion  Accuracy  Ratios  -  Analog  Equipments"  shows  a  comparison  between  the 
two  prediction  techniques. 

TABLE  VII 

ADJUSTED  PREDICTION  ACCURACY  RATIOS  VS  RADC  VOLUME  II 
PREDICTION  ACCURACY  RAFIOS  -  ANALOG  EQUIPMENTS 


Equipment 
Code  No. 

rAkT~ 

Ratio 

Adjusted 

Ratio 

2 

26.5 

12.5 

3 

13.4 

7.1 

4 

0.8 

0.5 

11 

3.8 

2.3 

25 

0.5 

4.2 

26 

10.2 

7.0 

27 

0.5 

0.3 

28 

0.9 

0.5 

30 

1.5 

1.0 

31 

0.9 

1.1 

34 

2.3 

1.0 

The  mean  for  the  RADC  Volume  II  prediction  is  5.6  with  a  standard  devia¬ 
tion  of  8.2.  The  mean  for  the  adjusted  prediction  is  3.5  with  a  standard 
deviation  of  3.9.  Thus,  the  adjusted  predictions  represent  a  60  percent 
increase  in  accuracy  and  a  110  percent  improvement  in  precision  for  the 
analog  equipments. 
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In  stMury*  the  results  of  this  study  have  shown  that: 

o  The  various  prediction  techniques  have  a  high  degree  of  linear 
correlation. 

o  The  prediction  techniques,  generally,  do  not  correlate  with  the 
observed  data. 

o  The  prediction  techniques  indicated  good  correlation  with  the 
observed  data  for  digital  equipments. 

o  The  KTBF  ratios  exhibit  a  log-normal  distribution. 

o  The  different  prediction  techniques  resulted  in  equal  variances  but 
different  meai.*>  which  indicates  that  a  difference  in  accuracy  exists 
between  techniques  but  there  is  no  difference  in  precision. 

o  The  analog  and  digital  equipments  were  not  of  the  same  log-normal  popu¬ 
lation  (i.e.,  the  means  and  variances  were  not  equal). 

o  The  technique  for  predicting  digital  equipment  reliability  should  be 
different  from  the  technique  used  for  predicting  analog  equipment 
reliability. 

o  Stress  analysis  predictions  produce  lower-than-observed  MTBF's  for 
almost  all  equipments  In  the  digital  design  approach  category. 

o  Power  equipment  demonstrated  characteristically  a  higher  than  desired 
(i.e.,  >1.0)  prediction  accuracy  ratio. 

o  Control  equipment  exhibited  the  widest  range  of  prediction  accuracy 
ratios. 

o  Vacuum  tube  equipments  were  characterized  by  a  lower  than  desired  (i.e., 
<1 .0)  prediction  accuracy  ratio. 

o  A  relationship  was  indicated  to  exist  between  the  accuracy  of  reliabil¬ 
ity  predictions  for  large  systems,  and  the  effects  of  program  factors 
(e.g.,  strength  of  reliability  programs).  System  prediction  modifiers 
and  program  rating  criteria  were  established. 

o  A  linear  function  provides  the  modifiers  to  improve  the  RADC  Volume  II 
predictions  for  both  digital  and  analog  equipments.  These  modifiers 
improve  both  the  accuracy  and  the  precision  of  the  predictions. 
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3.  DETAILS 


a.  Technical  Approach 

This  portion  of  the  study  is  an  attempt  to  assess  the  accuracy  of  the 
stress  analysis  prediction  techniques,  isolate  the  factors  which  contri¬ 
bute  to  the  inaccuracies  of  the  technique,  and  develop  a  more  accurate 
predict •','n  technique. 

The  solution  to  the  prediction  inaccuracy  problem  was  approached  by  se¬ 
lecting  a  group  of  electronic  equipment,  operating  in  a  fixed  ground 
environment  and  establishing  predicted  reliability  values  (i.e.,  RADC 
Volume  II,  upper  and  lower  grade  parts,  MIL  HDBK  217A,  and  Original  Con¬ 
tractor  predictions)  for  each  of  the  selectee  equipments. 

The  stress  analysis  prediction  techniques  were  evaluated  to  determine 
their  correlation  with  each  other  and  with  the  observed  data,  the  degree 
of  accuracy  of  the  individual  techniques,  and  the  probability  distribu¬ 
tions  formed  by  the  <atio  of  observed  mear.-time-be tween  failure  {MTBF0) 
to  the  predicted  mejn-tiine-between  failure  (MTBFp)  for  each  equipment. 

The  new  prediction  technique  was  validated  by  comparison  of  prediction 
accuracy  ratios  between  the  new  technique  and  the  RADC  Volume  II  technique. 

The  evaluation  of  stress  analysis  prediction  techniques  required  the  es¬ 
tablishment  of  predictions  which  utilized  the  techniques.  Therefore, 
predictions  were  prepared  for  thirty-eight  equipments  using  both  the 
methods  of  the  RADC  Reliability  Notebook,  Volume  II  and  MIL-HDBK-217A. 

The  stress  analysis  prediction  techniques  require  a  knowledge  of  numerous 
design  and  operating  factors  which  influence  the  failure  rates  of  parts 
within  the  equipment  (e.g.,  temperature,  stress,  environment,  part  quality, 
etc.).  Therefore,  information  relative  to  the  operating  characteristics 
of  the  equipment  were  obtained  from  the  following  sources: 

o  System  Environmental  Specifications 

o  Environmental  Control  System  Project  Engineers 

o  Equipment  Designers 

o  Contractor  Reliability  Prediction  Reports 

Utilizing  the  above  information,  a  standard  set  of  prediction  factors  and 
operating  conditions  was  established  for  use  in  preparing  all  R/.DC  Volume 
II  type  stress  analysis  predictions  as  listed  below. 

o  The  equipment  operates  in  a  fixed  ground  environment. 

o  The  part  quality  level  is  consistent  throughout  the  equipment  (i.e., 
upper  grade  or  lower  grade). 


'  -  — <  -1-.L 
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The  general  equation  used  for  establishing  a  part  failure  rate  for  the 
RADC  Volume  II  prediction  is 

=  *b  x  T  ^E’  (2) 

where 

x„  =  the  part  failure  rate, 

P 

x.  =  the  base  failure  rate  established  for  temperature, 

D  stress,  and  part  quality, 

nM  =  the  application  modifiers,  and 
Eg  =  the  environment  modifiers, 
b.  Correlation  Analysis 

This  study  evaluated  the  correlation  of  Mean-Time-Between-Failure  (MTBF) 
values  predicted  by  each  of  the  stress  analysis  prediction  techniques  with 
observed  values  and  each  other.  The  purpose  of  this  study  was  to  statisti¬ 
cally  evaluate  the  relationships  of  prediction  techniques  for  a  variety  of 
equipment,  and  to  determine  if  any  combinations  of  techniques  and  equipment 
exhibited  relationships  significant  to  the  evaluation  and  improvement  of 
prediction  accuracy,  precision,  or  repeatability.  The  study  also  attempted 
to  identify  and  isolate  any  significant  statistical  differences  which  might 
exist. 

The  test  for  simple  linear  correlation  was  performed  for  three  equipment 
categories: 

o  All  equipments  (34  equipments), 
o  Analog  equipments  (20  equipments),  and 
o  Digital  equipments  (14  equipments). 

The  correlation  coefficients  obtained  for  the  different  prediction  tech¬ 
niques  are  shown  in  Table  VIII,  "Simple  Linear  Correlation  Coefficients 
Between  Stress  Analysis  Prediction  Techniques." 
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TABLE  VIII 

SIMPLE  LINEAR  CORRELATION  COEFFICIENTS 
BETWEEN  STRESS  ANALYSIS  PREDICTION  TECHNIQUES 


Equipment  Type 

RADC 

Upper 

Grade 

RADC 

Lower 

Grade 

H 

All 

RADC  Upper  Grade 

— 

0.913 

RADC  Lower  Grade 

0.913 

— 

Analog 

RADC  Upper  Grade 

-- 

0.923 

0.986 

RADC  Lower  Grade 

0.923 

— 

0.917 

Digital 

RADC  Upper  Grade 

-- 

0.958 

0.722 

RADC  Lower  Grade 

0.958 

0.829 

The  foregoing  table  shows  relatively  high  degree  of  linear  correlation 
between  the  stress  analysis  prediction  techniques  (i.e.,  the  results  of 
all  stress  analysis  prediction  techniques  are  quite  similar).  Although 
this  correlation  provides  no  insight  into  tne  accuracy  of  the  techniques, 
it  shows  that  the  precision  of  the  individual  techniques  is  essentially 
the  same  for  all. 

The  correlation  analyses  conducted  between  the  observed  data  and  the 
several  stress  analysis  prediction  techniques  are  illustrated  in  Table  IX, 
"Correlation  Coefficients  Between  Observed  Data  and  Stress  Analysis  Pre¬ 
diction  Techniques." 

The  table  shows  a  significant  correlation  between  predicted  and  ob¬ 
served  values  for  digital  equipment,  but  very  little  correlation 
between  the  predicted  and  observed  values  for  analog  equipments. 

This  implies  that  a  significant  difference  exists  between  analog  and 
digital  equipment  which  is  not  compensated  for  in  the  prediction 
techniques. 
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TABLE  IX 

CORRELATION  COEFFICIENTS  BETl£En  OBSERVED  DATA 


it  STRESS  ANALYSIS  PREDICTION  TECHNIQUES 


Equipment  Categories 

RADC  |  RADC 

Upper  i  Lower 

Grade  j  Grade 

i 

• 

NIL-  • 

HOBK  i 

217A  ! 

) 

i 

All  Equipments 

1 

0.520  |  0.336 

i 

0.208  j 

i 

Analog  Equipments 

0.269  j  C-.192 

i 

0.132  i 

» 

L  1 

Digital  Equipments 

1 

0.963  !  0.868  i  0.604  1 

_ i _ i _ _  1 

The  high  correlation  (0-963)  between  the  RADC  Volume  II  upper  grade 
prediction  and  the  observed  data  for  the  digital  equipment  precipi¬ 
tated  a  least-squares  curve  fit  to  determine  what  curve  would  best 
fit  the  observed  data.  The  curves  used  in  the  least-squares  curve 


fit 

are: 

0 

y  =  a  +  B  x. 

0 

Sx 

y  *  ae  , 

0 

t* 

,  D 

y  =  ax  , 

0 

y 

0 

1-  1 
*  a  +  BX. 

c 

y  -  x 

a  +  Bx 

where 


y  3 

the 

x  = 

the 

5,6  = 

the 

observed  MTBF, 
predicted  KTBF,  and 
least-squares  coefficients. 
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The  results  of  the  least-squares  curve  fit  are  shown  in  Table  7,  "Least 
Squares  Curve  Fit,  Digital  Equipment".  This  Table  shows  that  the  linear 
curve  is  the  best  fit  (i.e.,  the  linear  curve  has  the  highest  index  of 
determination) .  However,  the  negative  value  of  o  is  undesirable.  The 
curve  where  y  is  a  power  function  (i.e.,  y  *  ay8}  is  the  second  best  fit. 
In  fact,  this  curve  indicates  that  a  awltiplier  of  aoproxisately  0.32 
would  sake  the  RADC  Voli»e  II  upper  grade  prediction  very  close  for  digi¬ 
tal  equipaents. 


TABLE  X 


LEAST-SQUARES  CURVE  FIT 
DIGITAL  EQUIPJCHTS 


r 

Curve 

1  _ 

Index  Of 
Determination 

• 

8 

j 

y  =  a  +  Sx 

G.93 

-14684.9 

i 

! 

7.09  i 

1 

T 

sx 

y  =  ae 

0.59 

1937.7 

0.0001 

y  =  coT 

0.90 

0.32 

1.25 

y  =  a  +  3/x 

0.13 

66144.8 

-2.0  x  10"7- 

y  =  l/a+Sx 

0.15 

0.0014 

-5.0  x  10'8 

y  =  x/a+Sx 

0.59 

0.84 

-6.2  x  10‘5 
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Using  the  least-squares  curve  fi  „  fficients  from  Table  X  for  the 
power  function  curve,  new  predict. ons  were  Bade  for  the  fourteen 
digital  equipments.  The  aquatic-:  for  the  new  prediction  is: 

y  *  axe, 

where 

y  =  the  new  prediction  value, 

x  =  the  old  RADC  Volume  II  upper  grade  prediction  value,  and 

c  ,8  =  the  least-squares  coefficients  (Table  X). 

Table  XI,  "Digital  Equipment  Prediction  From  Least- Squares  Coeffi¬ 
cients",  shows  the  results  of  predictions  for  the  digital  equipments 
utilizing  the  least-squares  curve  fit  coefficients. 

TABLE  XI 

DIGITAL  EQUIPMENT  PREDICTIONS 
FROM  LEAST-SQUARES  COEFFICIENTS 


Equipment 
Code  No. 

Prediction 
(y  =  ax*} 

Observed 

L.S.  Pred.  t 

.  i 

Observed 

RADC  Pred. 

37 

254.8 

! 

I 

4.2  1 

7.1 

20 

389.3 

0.5 

C.9 

32 

432.5 

4.2 

11.4  i 

23 

654.3 

0.9 

1.8 

19 

1510.3 

0.2 

0.7 

18 

3164.5 

0.3 

1.7 

1 

6854.5 

0.7 

1.7 

6 

7632.1 

1.6 

3.8 

8 

24207.8 

1.2 

3.8 

10 

32493.1 

0.6 

2.1 

7 

37906.6 

2.1 

7.0 

5 

46071.6 

1.7 

6.1 

29 

51808.3 

0.4 

1.3 

9 

230061.0 

1.5 

7.3 

. I 

x  =  1.4 

1 

x  =  4.0 

The  ratios  of  the  observed  MTBF  to  the  new  predicted  MTBF  were  ob¬ 
tained.  These  ratios  indicate,  when  compared  with  the  ratios  of 
observed  MTBF  to  RADC  Volume  II  upper  grade  predicted  MTBF,  that  the 
least-squares  coefficients  give  a  mors  accurate  digital  equipment 
prediction  (i.e.,  x  is  closer  to  one)  but  do  not  improve  upon  the 
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precision  of  the  prediction  (i.e.,  the  range  of  variability  between  the 
ratios  for  all  equipments  is  essentially  the  same). 

c.  Analysis  of  MTBF  Ratios 

(1)  Introduction 

The  ratios  of  observed  MTBF  to  predicted  KTBF  were  calculated  for 
thirty-four  equipments  using  various  stress  analyses  prediction 
techniques.  These  ratios  are  shown,  by  equipment  numbers,  in  Table 
XII,  “Ratio  of  Observed  MTBF  to  Predicted  MTBF.” 

It  is  desirable  to  know  the  underlying  probability  distribution  of 
the  ratios  of  observed  MTBF  to  predicted  KTBF. 

If  the  probability  distribution  can  be  determined,  then  it  becomes 
possible  to: 

o  determine  if  the  prediction  techniques  produce  equal  means  and 
variances.  That  is,  is  any  one  prediction  technique  either  more 
accurate  or  more  precise  than  any  othe»  technique? 

o  determine  if  all  types  of  equipment  can  utilize  the  same  predic¬ 
tion  technique.  That  is,  do  all  types  of  equipment  have  t ha 
same  type  of  probability  distribution  and  if  so,  are  they  the 
same  distribution? 

Thus,  the  analysis  of  the  MTBF  ratios  is  accomplished  in  the  following 
manner: 

o  Determine  the  underlying  probability  distributions  for  the  differ¬ 
ent  types  uf  equipments  (i.e.,  digital,  analog,  and  all  equipments) 
via  "goodness  of  fit"  tests. 

o  Determine,  for  the  different  equipment  types  whose  ratios  have  the 
same  type  of  probability  distribution,  if  they  are,  indeed,  from 
the  same  distribution  via  tests  for  equal  means  and  equal  vari¬ 
ances.  That  is,  the  MTBF  ratios  from  two  or  more  equipment  types 
might  exhibit  a  normal  distribution,  but  also  have  different 
means  and  variances  and  hence,  would  not  be  members  of  the  same 
normal  distribution. 

(2)  Goodness  of  Fit  Tests 

These  tests  are  performed  to  determine  the  underlying  probability 
distributions  of  the  MTBF  ratios. 

The  categories  of  equipment  considered  in  the  MTBF  ratio  analysis  are: 
o  Digital  (14  equipments) 
o  Analog  (20  equipments) 
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TABLE  XII 


RATIO  OF  OBSERVED  MTBF 
TO  PREDICTED  MTBF 


Equipment 
Code  Number 

HIL-HDBK 

21 7A 

Prediction 

RADC 

Upper  Grade 
Prediction 

RADC 

Lower  Grade 
Prediction 

Original 
Contractor 
Predi  cti  on 

I 

8.8 

1.7 

10.8 

2 

51.4 

26.5 

255.0 

8.0 

3 

34.4 

13.4 

111.6 

7.2 

4 

2.7 

6.8 

5.4 

- * 

5 

17.8 

5.1 

53.8 

— 

6 

120. 2 

3.8 

50.5 

--- 

7 

57.5 

7.0 

53.9 

8 

50.7 

3.8 

36.0 

— 

9 

91.5 

7.3 

71.3 

— 

10 

52.2 

2.1 

24.6 

— - 

11 

19.9 

3.8 

25.4 

— 

12 

0.2 

0.06 

0.13 

0.16 

13 

0.15 

0.09 

0.23 

0.11 

14 

1.8 

0.9 

4.1 

0.4 

15 

1.0 

0.4 

3.4 

0.5 

16 

0.18 

0.05 

0.15 

0.31 

17 

0.14 

0.05 

0.27 

0.10 

18 

11.4 

1.7 

12.8 

3.2 

!  19 

5.8 

0.7 

6.2 

1.6 

i  20 

6.9 

0.9 

8.1 

1.5 

1  21 

2.5 

0.9 

7.1 

1.5 

22 

1.4 

0.4 

3.0 

0.7 

23 

5.0 

1.8 

12.3 

3.7 

25 

1.6 

0.5 

4.0 

0.9 

!  26 

50.7 

10.2 

56.2 

8.8 

;  27 

1.4 

0.5 

2.0 

0.4 

j  28 

1.9 

0.9 

3.2 

1.5 

1  29 

4.6 

1.3 

6.5 

1.4 

30 

4.9 

1.5 

10.5 

3.6 

31 

2.S 

0.9 

5.9 

2.3 

32 

36.3 

11.4 

111.4 

2.S 

34 

4.8 

2.2 

18.3 

2.2 

36 

0.3 

0.08 

0.4 

0.1 

37 

i _ 

27.0 

7.1 

67.9 

1.4 
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o  All  equipments  (34  equipments). 

Having  prior  knowledge  that  these  ratios  might  be  either  normally  or 
log-normally  distributed,  Chi-Square  and  Kolmogorov-Smirnov  "Goodness- 
of-Fit"  tests  can  be  applied  to  the  data  to  determine  the  underlying 
probability  distributions.  Therefore,  a  chi-square  goodness-of-flt 
test  is  first  applied  to  the  ratios. 

The  hypothesis  to  be  tested,  at  the  0.05  significance  level,  is 
whether  the  ratios  of  observed  MTBF  to  predicted  KTBF  for  any  or  all 
of  the  three  equipment  categories  have  an  underlying  probability 
distribution  which  is  either  normal  (Gaussian)  or  log-normal. 

Table  XIII,  "Chi-Square  Goodness  of  Fit  Test  for  Normal  Distribu¬ 
tion",  shews  tne  results  of  the  Chi-Square  test  for  a  normal  distri¬ 
bution  of  the  ratios. 

TABLE  XIII 

CHI -SQUARE  G000NESS  OF  FIT  TEST 
FOR  NORHAL  DISTRIBUTION 


i 

MIL-HDBK-21 

7A 

RADC 

Upper  Grade 

LC|u  i  pnen  t 

Category 

3an<p  1  c 

Size 

[  V- 

D.F. 

2 

x0 

'505,  V 

Signi¬ 

ficant 

x0 

2  Signi- 

505,v  ficant 

All  Equip. 

34 

6 

12.7 

7.82 

yes 

2.71 

7.82  no 

Analog  Equip. 

20 

i 

5.95 

3.84 

yes 

9.29 

3.84  yes 

Digital  Equip. 

14 

1 

4.67 

3.84 

1 

yes 

0.330 

3.84  no 

From  Table  XIII,  we  reject  the  hypothesis  that  the  ratios  of  ob¬ 
served  MTBF  to  predicted  MTBF  are  normally  distributed  when  the  pre¬ 
diction  is  made  via  the  MIL-HDBK-217A  techniaue.  Also,  we  reject 
the  hypothesis  for  the  analog  equipments  when  the  prediction  is 
formed  via  the  RADC  Volume  II  technique.  We  are  unable  to  state 
that  the  ratios  for  digital  and  all  equipment  categories  are  not 
normally  distributed  for  the  RADC  predictions.  However,  it  is  ob¬ 
viously  the  strong  influence  of  the  digital  equipment  which  prevents 
the  "all  equipments"  category  from  being  significant  (i.e.,  re¬ 
jected)  at  the  0.05  level. 

Table  XIV,  "Chi-Square  Goodness  of  Fit  Test  for  Log-Normal  Distribu¬ 
tion",  iTustrates  the  results  of  the  Chi-Square  test  for  a  log¬ 
normal  distribution  of  the  ratios. 


TABLt  XIV 

CHI-SQUARE  GOODNESS  OF  FIT  TEST  FOR 
LG6-NORMAL  DISTRIBUTION 


Equipment 

Category 

Sample 

Size 

n 

1 

1 

1 

LiiinsxsEm 

— j— 

*o 

mi 

rntma 

2 

*0 

UK 

13PIM 

Bffuil 

All  Equip. 

34 

4.59 

7.82 

no 

2.47 

7.82 

no 

Aiiclog  Equip. 

20 

1.53 

3.84 

no 

1.02 

3.84 

no 

Digital  Equip. 

14 

n 

0.084 

3.84 

no 

0.323 

3.84 

no 

In  Table  XIV  it  is  shewn  that  we  cannot  reject  the  hypothesi:  that 
the  ratios  of  any  of  the  equipment  categories  (i.e.,  all,  analog, 
or  digital)  are  log-normally  distributed  whether  the  prediction 
utilizes  the  MIL-HDBK-217A  or  the  RADC  technique. 

The  KolmogoroY-Smirnov  goodness  of  fit  test  was  applied  to  the 
ratios  of  the  same  equipment  categories  as  the  Chi-Square  test  to 
ascertain  whether  the  ratios  by  equipment  categories  were  normally 
or  log-normally  distributed.  The  Kolmogorov- Smirnov  Test  is  given 
by  the  expression 

Da,v  =  MAX j F  -  $n  |;  i  =  1,2, . n  =  number  of  cells,  (3) 

°i  ni 

where 

F  =  the  theoretical  cumulative  frequency  distribution  up 
°i  to  and  including  the  ith  cell, 

S  =  the  observed  emulative  frequency  distribution  up  to 
ni  and  including  the  i*"  cell, 

a  =  the  significance  level, 

v  2  the  degress  of  freedom,  2nd  equal  to  the  total  number 
of  observations. 


Table  XV,  "Kolmogorov-Smirnov  Test  For  Normal  Distribution," 
illustrates  the  results  of  this  test  by  equipment  categories. 


TABLE  XV 


KOLOMQGORO V-SMI RNOV  TEST 
FOR  NORMAL  DISTRIBUTION 


Equipment 

Category 

S'.mple 

Size 

v= 

D.F. 

M 

[L-HDBK-217A 

l 

RADC  Upper  Grade  1 

D0 

?05,v 

Significant 

Do 

Dgs  v  Significant 

All  Equipments 

34 

34 

0.254 

0.23 

yes 

0.261 

0.23  :  yes 

1 

\r.alog  Equip. 

20 

20 

0,361 

0.294 

: 

yes  ;  0.319 

i  . ...  . 

! 

0.294j  yes 

Digital  Equip. 

14 

14 

0.181 

0.349 

l 

i 

no  j  0.094 

_ i _ 

O.lBl!  no 

_ i _ 

From  Table  XV  we  can  reject  the  hypothesis  that  either  of  the 
categories  of  "all  equipment"  and  "analoy  equipment"  are  normally 
distributed  regardless  of  the  prediction  technique.  However,  we 
cannot  reject  the  hypothesis  that  the  digital  equipment  is  normally 
distributed. 

The  results  of  the  Kolmogorov-Smimov  test  for  the  log-normal  dis¬ 
tribution  are  shown  in  Table  XVI,  "Kolmogorov-Smimov  Test  For 
Log-Normal  Distribution". 


TABLE  XVI 

K0LM0G0R0V-SMIRN0V  TEST  FOR 
LOG-NORMAL  DISTRIBUTION 


—  1  —  ■  ■'  - 

Equipment 

Category 

Sample 

Size 

\)  = 

D.F. 

MIL-HDBK-21 

7A 

RADC  Upper  Grade 

H 

mm 

Significant 

m 

m 

Significant 

34 

34 

0.079 

0.23 

no 

0.091 

0.23 

no 

20 

20 

0.091 

0.294 

no 

O.OSS 

0.294 

no 

Digital  Equip. 

14 

14 

0.054 

0.349 

no 

0.109 

0.349 

no 

From  Table  XVI,  we  can  observe  that  we  cannot  reject  the  hypothe¬ 
sis  that  the  ratios  for  equipment  categories  have  log-normal  dis¬ 
tributions  regardless  of  the  prediction  technique. 
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fro"  Chi-Square  and  the  Kolmogorov-Smirnov  test 

observed  nFtMSffm?  '*  "*  **"  fit  for  tte  rat1os  « 

(3)  Tests  for  Equal  Variances  and  Means 

Since  there  appears  to  be  a  difference  between  the  digital  and  analog 
equipments  rtiich  way  require  different  prediction  techniques  for  the9 
two  equipment  types,  further  tests  were  employed  to  determine  if  the 
equipments  were  part  of  the  same  log-normal  distribution.  These  tests 

o  Bartlett's  Humber  (a  test  for  the  equality  of  variances), 
o  F  Ratio  Test  (a  test  for  the  equality  of  means), 
o  *t"  test  (a  test  for  the  equality  of  means). 

Prerequisites  for  the  analog  and  digital  equipments  to  be  momhor* 

>T"0r'MLPT,a;i0"'  that  thelrwneans'anlT variances^ 
e  equal c  Hence,  the  first  step  was  to  determine  if  the  varianrec 

useful  test  for  determining  if  the  variances  are  eoual 

"Ba^tlPtt^VTont^>rerJhao  tW?  variances  involved,  is  the  q  ’ 

arithmetic^veraae  of^h^r^'5  t?st  iS  3  comPari*sor.  between  the 
avera?e  of  the  sample  variances  ana  the  geometric  aver¬ 
age  of  those  variances.  The  geometric  average  i$  less  sensitive 

jLef!^irfues  mn  is  the  arithmetic  average.  Hence  a  ratio  of 
vafn^wh^"6^0  ave^a9e  ^  the  geometric  average  produces  a  large 
VaIi<e  W^en  var^ance  is  non-uniform,  and  conversely  the  ratio 
produces  a  small  value  when  the  variances  are  unifom 

ftdence>levelB(r,)ttwithUfhrrK?  ?*>,ared>  at  the  appropriate  con- 

freed«  irrJhr^r^^p^^^r^s^ :eos^f  ?hr:ecfif 

KoSrL%"rber  excee5s  thd  Ch1-Csequ^Svrrlat^theneoC„r 

may  conclude  that  toe  variances  are  not  equal. 

d?Ii*ii0nS,Were  dfitermined  and  used  to  compute  the 

tS  f0r  S  X  ca!es:  Ttlese  six  cases  Wf,re  designed  to 

^  Ifariances  the  ratios  within  the  three  equipment 
categories c  and  also  test  between  the  variances  of  the  ratios  between 
equipment  categories.  The  six  cases  and  the  results  of  these  tests 

Variances!*"^  Tab1e  ’  "Bartlett's  Test  for IqSaliSy  of 
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TABLE  XVII 
BARTLETT'S  TEST 
FOR  EQUALITY  OF  VARIANCES 


Equipment 

Category 

No.  of 

Sample 

Variances  (k) 

Bartlett's 

Number 

2 

*05^-] 

_  i 

Test 

Case 

dumber 

Significant 

All  Equipment 

4 

5.26 

7.82  I  1 
» 

! 

No 

Analog  Equipment 

4 

2.41 

j 

7.82  |  2 

No 

_ l 

Digital  Equipment 

4 

5.65 

7.82  (  3  |  No 

All  vs  Analog 

8 

7.83 

— 

14.07 

. 

4 

No 

All  vs  Digital 

8 

30.72 

14.07 

5 

Yes 

Analog  vs 

Digital 

8 

28.36 

1 

14.07 

6 

_ 

Yes 

- - —  J 

From  the  above  table.  It  is  evident  that  the  variances  within  the 
three  equipment  categories  may  be  considered  equal.  In  addition, 
the  variances  between  the  all  equipment  category  and  the  analog 
equipment  category  may  be  considered  equal.  However,  the  variance 
associated  with  the  digital  equipment  category  is  different  from  the 
variances  of  the  other  equipment  categories. 


The  next  step  in  determining  if  the  log-normal  populations  are  one 
and  the  same  for  analog  and  digital  equipments,  is  to  deteimine  if 
the  means  of  the  populations  are  equal.  The  test  for  the  equality 
of  population  means  is  based  on  the  analyses  of  variance  technique. 
This  technique  (the  F  ratio  test)  is  based  on  the  assumption  that 
the  population  variances  are  equal.  '  ' 

The  value  of  F  which  is  calculated  is  tested  against  a  tabular  value 
of  F  with  K-l  and  N-K  degrees  of  freedom  (K  is  the  number  of  means 
under  test  and  N  is  the  sum  of  the  observation  for  each  sample.)  and 
at  the  desired  confidence  level.  If  the  calculated  value  exceeds  the 
tabular  value  of  F,  one  may  conclude  with  (1-a)  percent  confidence 
that  the  means  are  unequal. 
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The  F  ratio  test  was  performed  on  the  three  equipment  categories 
where  the  variances  within  equipments  were  equal.  The  results  of 
the  F  ratio  test  are  shown  in  Table  XVIII, "F  Ratio  Test  For  The 
Equality  of  Means’*. 


TABLE  XVIII 
F  RATIO  TEST 

FOR  THE  EQUALITY  OF  MEANS 


Equipment 

Category 

Fo 

h05;K-l ,N-K 

Significant 

F01 ;K-1 ,N-K 

Significant 

All 

Equipments 

10.47 

2.68 

Yes 

3.94 

Yes 

Analog 

Equipments 

3.66 

2.73 

Yes 

4.06 

No 

Digital 

Equipments 

21.21 

2.82 

Yes 

4.25 

Yes 

From  the  above  Table,  we  recognize  that  we  can  reject  the  hypothe¬ 
sis  of  equal  means  for  all  three  equipment  categories  with  95  per¬ 
cent  confidence.  Also,  the  hypothesis  of  equal  means  within  the 
equipment  categories  can  be  rejected  with  99  percent  confidence  for 
all  categories  except  the  analog  equipment  category.  This  indicates 
that  different  log-normal  distributions  exist  in  each  case. 

Since  the  test  for  equal  variances  between  equipment  categories 
showed  a  definite  difference  between  the  analog  equipment  variance 
and  the  digital  equipment  variance,  the  F  ratio  test  could  not  be 
utilized  to  determine  the  equality  of  the  means  between  these  two 
equipment  categories.  Hence,  a  Student's  "t"  test  was  used  to  test 
the  hypothesis  of  equal  means  between  analog  equipment  and  digital 
equipment. 

The  computed  value  of  ,'t"  is  compared  with  a  tabular  value  of  "t" 
at  the  desired  confidence  level  and  the  appropriate  degrees  of 
freedom.  There  were  four  comparison  cases  determined  between  the 
means  of  analog  and  digital  equipments  (i.e.,  MIL-HDBK-217A,  RADC 
Volume  II  Upper  Grade,  RADC  Volume  II  Lower  Grade,  and  Original 
Contractor  predictions).  The  results  of  these  tests  are  shown  in 
Table  XIX,  '  "t"  Test  For  The  Equality  of  Means.' 


TABLE  XIX 


"t“  -  TEST  FOR 
EQUALITY  OF  MEANS 


Prediction 

D.F. 

*o 

t.01 

Significant 

t.05 

Significant 

MIL-HDBK.-217A 

32 

4.49 

2.75 

Yes 

2.04 

Yes 

RADC  Upper  Grade 

29 

3.00 

2.76 

Yes 

2.05 

Yes 

RADC  Lower  Grade 

28 

5.35 

2.76 

Yes 

2.05 

Yes 

Original  Contractor 

22 

2.31 

2.82 

No 

2.07 

Yes 

From  the  above  table,  it  can  be  noted  that  we  reject  the  hypothesis 
of  equal  means  between  analog  and  digital  equipment  ratios  for  all 
prediction  techniques  with  95  percent  confidence.  However,  we  can¬ 
not  reject  the  hypothesis  of  equal  means  between  analog  and  digital 
equipment  ratios  for  the  original  contractor  prediction  with  99  per¬ 
cent  confidence.  The  above  inoi cates  that  in  all  cases  the  analog 
and  digital  equipments  are  from  different  log-normal  distributions. 
Thus,  if  accuracy  is  to  be  increased  in  prediction  techniques ftnen 
analog  and  digital  equipments  require  different  techniques. 

The  reliability  prediction  (i.e.,  RADC  Volume  II  upper  grade  parts) 
were  divided  into  categories  (i.e.,  high  MTBF,  low  MT8F,  close 
agreement  to  observed  MTBF,  and  those  for  which  insufficient  ob¬ 
served  data  are  available).  The  following  tabulations  evaluate 
the  divisions  of  the  equipments.  (See  Table  XX  -  Table  XXII) 


TABLE  XX 


EQUIPMENTS 

WITH  HIGH  MTBF  RATIOS 


Equipment 
Code  Number 

Ratio 

MTBF  /MTBF„ 
o  p 

Analog  or 
Digital 

Function 

Percent 

Active  Elements 

2 

26.5 

Analog 

Power 

43.0 

3 

13.4 

Analog 

Power 

29.0 

5 

6.1 

Digital 

Control 

23.4 

6 

3.8 

Digital 

Computer 

61.2 

7 

7.0 

Digital 

Amplifier 

37.8 

8 

3.8 

Digital 

Control 

55.1 

9 

7.3 

Digital 

Control 

35.2 

10 

2.1 

Digital 

Computer 

53.3 

11 

3.8 

Analog 

Power 

28.9 

26 

10.2 

Analog 

Power 

32.2 

32 

11.4 

Digital 

Computer 

36.5 

34 

2.2 

Analog 

Display 

30.2 

37 

7.1 

Digital 

Computer 

41.2 

There  were  eight  digital  equipments  and  five  analoq.  equipments  whose 
MTBF  ratios  were  high  (i.e.,  >2.0).  The  five  analog  equipments  con¬ 
tain  four  equipments  whose  function  is  power  (e.g.,  power  supply) 
and  of  these  four  power  equipments  there  were  three  equipments  with 
extremely  high  ratios  (i.e.  >10.0).  In  fact,  these  three  equip¬ 

ments  were  on  the  extreme  end  of  the  high  MTBF  scale.  Also,  the 
table  indicates  that  more  digital  equipment  predictions  under-esti¬ 
mate  the  equipment  MTBF's  than  analog  equipment  predictions.  Pre¬ 
dominant  functional  groups  are  computers  and  control  equipments  (seven 
of  the  eight  digital  equipments).  There  is  no  evident  relationship 
between  the  percent  of  active  elements  and  the  MTBF  ratios. 

There  were  nine  equipments  whose  MTBF  ratios  were  low  (i.e.,  <0.5) 
These  nine  equipments  were  all  analog  equipments.  Table  XXI  "Analog 
Equipments  with  Low  MTBF  Ratios,"  lists  these  nine  equipments,  their 
MTBF  ratio,  function,  and  percent  of  active  elements. 
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TABLE  XXI 
ANALOG  EQUIPMENTS 
WITH  LOW  MTBF  RATIOS 


Equipment 
Cede  Number 


Approximately  5  Percent  Tubes 


The  above  table  indicates  a  gross  relationship  between  the  inci¬ 
dence  of  active  elements  (tubes)  and  the  very  low  (i.e.,  <  0. 1 ) 
MTBF  Ratio.  The  four  tube  type  equipments  have  the  lowest  ratios 
and  the  lowest  percent  active  elements.  However,  because  of  the 
small  sample,  a  definite  conclusion  on  the  relationships  could  not 
be  established.  From  Table  XX,  "Equipments  with  High  MTBF  Ratios", 
and  Table  XXI  above,  it  is  evident  that  the  MTBF  ratios  for  equip¬ 
ments  with  the  control  function  vary  across  the  spectrum  of  vari¬ 
ability.  This  indicates  that  the  prediction  technique  for  control 
equipment  exhibits  a  wide  range  of  variation. 

Table  XXII,  "Equipments  with  Accurate  MTBF  Ratio",  show  the  twelve 
equipments  whose  MTBF  Ratios  were  accurate  (i.e.,  close  to  the 
observed  value). 


MTBF 

Ratio 

Function 

Percent 

Active  Elements 

0.07 

Control 

17.2* 

0.09 

Receiver 

15.2* 

0.40 

Control 

31.2 

0.06 

Control 

7.0* 

0.06 

Test 

29.3* 

0,40 

Control 

31.0 

0.50 

Control 

34.2 

0.46 

Control 

41.8 

0.08 

Test 

31.2 

TABLE  XXII 

EQUIPMENTS  WITH  ACCURATE 
MTBF  RATIOS 


Equipment 
Code  Nuaber 

KTBF 

Ratio 

Analog  or 
Digital 

Function 

Percent 

Active  Elements 

1 

1.71 

Digital 

Computer 

46.0 

4 

0.76 

Analog 

Control 

26.4 

14 

0.88 

Analog 

Control 

23.7 

18 

1.68 

Digital 

Control 

42.8 

19 

0.69 

Digital 

Control 

46.4 

20 

G.88 

Digital 

Control 

47.7 

2T 

0.92 

Anal og 

Control 

33.4 

23 

1.78 

Digital 

Computer 

36.7 

28 

0.93 

Analog 

Display 

33.3 

29 

1.34 

Digital 

Power 

44.0 

30 

1.47 

Anelog 

Power 

40.0 

31 

0.94 

Analog 

Test 

50.1 

From  Table  XXII,  it  is  again  evident  that  no  correlation  exists 
between  MTBF  ratio  and  percent  of  active  elements.  Also,  the 
digital  and  analog  equipments  are  equally  represented. 
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IMPROVEMENT  OF  PRE-DESIGN  PREDICTION  TECHNIQUES 


1.  OBJECTIVES 

The  objectives  of  the  study  effort  concerned  with  improvement  of  pre¬ 
design  prediction  techniques  are  listed  below: 

o  Research  existing  pre-design  prediction  techniques  to  determine  areas 
wherein  improvement  should  be  made. 

o  Measure  the  accuracy  of  existing  pre-design  prediction  techniques. 

o  Through  the  use  of  operational  data,  develop  functional  modifiers  which 
can  be  used  to  provide  a  more  accurate  prediction. 

2.  SUMMARY 

Two  functions  were  developed,  which  will  improve  the  accuracy  of  pre¬ 
design  prediction  techniques  for  analog  power  and  control  equipments.  No 
function  was  found  to  have  a  sufficiently  high  correlation  coefficient  to 
enable  improved  pre-design  prediction  techniques  to  be  developed  for 
digital  electronic  equipment  and  display  equipment. 

The  two  techniques  which  were  developed  to  improve  the  pre-design  predic¬ 
tions  are  modifier  functions  to  the  MIL-STD-756A  prediction,  and  are  shown 
below  for  each  category  of  analog  equipment. 

Power 

MTBF  =  1958  (MT3F)0,73 
Control 

MTBF  =  ( MTBF)/ ( .0124  +  .00037  MTBF) 

Where 

MTBF  =  Modified  MTBF  prediction 
MTBF  =  MIL-STD-756A  Predicted  MTBF 

The  percent  difference  of  observed  to  improved  MTBF  predictions  are  shown 
in  Table  XXIII.  All  improved  predictions  are  considered  better  than  the 
original  MIL-STD-756A  prediction  with  one  exception;  namely,  equipment 
number  16,  where  the  modified  prediction  is  comparable  to  the  MIL-STD-756A 
prediction,  but  less  accurate.  The  predictions  for  power  equipment  are 
more  accurate  than  those  for  the  control  equipment,  with  all  predictions 
within  one  hundred  percent  accuracy. 
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TABLE  XXIII 


In  suwnary,  the  following  conclusions  were  made  as  a  result  of  the  analysis 

of  pre-design  prediction  techniques: 

o  All  pre-design  techniques  predict  lower-than-cbserved  MTBF  values  for 
most  electronic  equipment. 

o  Equipment  in  the  digital  and  analog  design  approach  categories  exhibited 
distinctly  different  ranges  of  prediction  accuracy  ratios. 

o  The  digital  design  approach  category  exhibited  a  positive  correlation 
between  prediction  accuracy  ratios  based  on  the  HIL-STD-756A,  prediction 
technique. 

o  The  analog  design  approach  category  failed  to  show  any  significant  cor¬ 
relation  between  the  prediction  accuracy  ratios  of  the  prediction 
techniques. 

o  Significant  differences  in  the  range  of  prediction  accuracy  ratios  were 
established  between  functional  categories  of  analog  equipment. 

o  Prediction  accuracy  can  be  improved  for  the  pre-design  prediction 
techniques  for  selected  functional  categories  of  analog  eouipment,  by 
the  use  of  a  functional  modifier. 

o  When  the  analog  equipment  is  subdivided  into  power,  control,  and  dis¬ 
play  groups,  a  positive  correlation  exists  between  the  WTBF  values 
produced  by  the  pre-design  prediction  techniques  and  measured  values 
for  each  group. 
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3.  DETAILS 

a.  Technical  Approach 

The  establishment  of  Improvements  In  current  predesign  prediction 
techniques  for  ground  electronic  equipment,  involved  an  analysis  of  the 
characteristics  of  the  prediction  techniques.  This  included  investi¬ 
gations  of  the  areas  where  the  techniques  were  valid,  and  an  identi¬ 
fication  of  the  weaknesses  of  the  techniques.  KTBF  predictions  for  the 
candidate  equipments  were  made,  using  the  existing  predesign  prediction 
techniques.  The  accuracy  of  the  predictions  was  evaluated  and  compared 
to  the  observed  equipment  MTBF,  using  the  ratios  of  observed  MTBF's  to 
predicted  MTBF's.  To  determine  whether  any  technique  could  be  improved 
to  more  accurately  predict  the  observed  MTBF  values, it  was  necessary  to 
determine  whether  or  not  the  technique  produced  values  having  a  funda¬ 
mental  relationship  to  the  observed  values.  For  this  purpose,  a 
regression  analysis  was  performed  on  the  predicted  and  observed  MTBF 
values  to  obtain  a  functional  fit  of  the  predicted  to  observed  MTBF 
values.  Six  functions,  listed  below, were  evaluated  in  making  the  best 
fit.  The  correlation  coefficient  was  used  to  indicate  the  goodness  of 
fit  for  each  function. 


0 

Y 

*  A  +  BX 

o  Y  = 

A  +  (B/X) 

0 

Y 

-  AeBX 

o  Y  = 

1/(A  +  BX) 

0 

Y 

*  AX8 

o  Y  = 

X/(A  +  BX) 

The  function  identified  as  having  the  best  fit  was  used  to  obtain  new 
predicted  MTBF  values  for  each  equipment.  The  new  MTBF  values  were 
compared  to  the  observed  values  and  a  distribution  analysis  of  the  ratios 
of  observed  to  predicted  MTBF  was  performed.  The  analyses  were  used  to 
identify  and  isolate  factors  causing  prediction  inaccuracy,  and  to 
determine  distribution  parameter  values  so  that  measures  of  the  inaccuracy 
o*  selected  prediction  techniques  could  be  determined. 

b.  Pre-Design  Reliability  Prodiction  Techniques 

(1)  Introduction 

Current  reliability  prediction  techniques  are  used  during  the  pre- 
design  phase  of  the  development  cycle.  These  techniques  are  based 
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upon  established  relationships  between  equipment  reliability  values 
(e.g.,  MTBF)  and  selected  equipment  characteristics  such  as: 

(1)  Equipment  function, 

(2)  Active  element  group  count, 

(3)  Power  consumption,  and 

(4)  Number  of  CRT  tubes. 

Each  of  the  techniques  evaluated  is  discussed  in  the  sections  below. 

(2)  Prediction  by  Function  -  Federal  Electric  Method 

The  prediction  by  function  technique  developed  by  the  Federal  Electric 
Corporation  was  based  upon  radar,  conmuni cation,  and  Electronic  Data 
Processing  (EDP)  central  processing  functions  operating  in  a  fixed 
ground  environment. 

The  communication  function  is  characterized  in  three  separate  sub¬ 
functions  as  follows:  "receive,"  "transmit,"  and  "multiplex."  For 
these  sub-functions,  respectively,  MTBF  can  be  predicted  as  a  function 
of  Noise  Figure  (DB),  Power  Gain  (DB)  and  the  number  of  voice 
channels  utilized,  using  the  following  expressions: 

WTBF^  a  =  (2889)  e~*136  NF 
recei ve 

^transmit  '  (6^9)  G" 624 
Multiplex  *  <783>  *"0mC 
Where, 

Np  =  Noise  figure  (in  decibels)  i.e., 

Slgnal-to-Noise  power  ratio  of  ideal  receiver _ 

Actual  Signal-to-Noise  power  ratio  of  receiver  output 

G  Power  gain  (in  decibels)  i.e.. 

Typical  average  power  output  of  final  amplifier 
Input  power  required  to  drive  final  amplifier 

C  =  Number  of  Communication  Channels 
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The  EDP  central  processor  prediction  technique  predicts  MTBF  as  a 
function  of  the  ratio  of  word  size  (In  bits)  to  add  time  (in  micro¬ 
seconds).  The  equation  for  the  EDP  central  processor  MTBF  prediction 
is: 


MTBF 


antral  processor 


(524) 


-.135  W/A 


where, 

W  ■  Word  size  in  bits 
A  3  Add  time  in  microseconds 

The  radar  function  is  characterized  in  two  separate  sub-functicns: 
"receiver,"  and  "transmitter."  For  these  sub-functions,  failure  rates 
per  active  element  are  obtained  through  the  following  functions: 

Ar  «  4.17  (P)*32 

At  «  9.06  (P)*36 

where* 

xr  *  Receiver  failure  rate  per  active  element  group 
At  *  Transmitter  failure  rate  per  active  element  group 

P  »  Peak  power  in  kilowatts. 

An  active  element  group  is  defined  as  a  tube  and  associated  circuitry, 
or  a  transistor  or  diode  and  its  associated  circuitry.  The  FEC  radar 
prediction  by  function  has  an  alternate  predictor  which  combines  the 
two  sub-functions  of  receive  and  transmit  into  a  single  equation  as 
follows: 


Vt  =  6.3(P)  ’3 


where, 

A  .  3  failure  rate  per  active  element  group  for  the  radar 
receive- transmit  function. 
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For  the  range  of  parameter  values  represented  for  radar,  coaminicatton 
and  central  processor  equipment,  the  maximtsn  and  minimtXR  MTBF  values 
and  their  respective  parameters  for  the  six  predictors  are  show,  in 
Table  XXIV.  The  functional  parameter  values  represent  the  range  of 
values  for  which  the  FEC  predictors  hold. 


TABLE  XXIV 

MINIMUM  AND  MAXIMUM  PREDICTION 
PARAfCTER  AND  MTBF  VALUES 


Predictor 

— 

Function 

Parameter  Value 

MTBF  Prediction 

Function 

Parameter 

Minimum 

Maximum 

Maximum 

.  . 

Mirimum 

Communication 
Recei ve 

Noise  Fig 

1  db 

11  db 

- 

2500  hrs 

60C  hrs 

Communication 

Transmit 

Power  Gain 

15  db 

75  db 

13UC  hrs 

500  hrs 

Communication 

Multiplex 

#  Voice 
Channels 

5 

45 

750  hrs 

350  hrs 

EDP  Central 
Processor 

Word  Size 

Add  Time 

.1  bit/uSec 

S  bit/uSec 

580  hrs 

160  hrs 

Radar 

Recei ver 

Peak  Power 

:  KW 

5000  KW 

240,000* 

13,000* 

Radar 

Transmitter 

Peak  Power 

1  KW 

5000  KW 

125,000* 

5,000* 

* 


MTBF  per  active  element 
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(2)  Prediction  by  Function  -  ARINC  Research  Corporation  Methods 


The  prediction  by  function  developed  by  ARINC  was  developed  by 
grouping  all  the  data  into  one  group,  in  contrast  to  the  three 
separate  groups  of  the  FEC  technique,  discussed  previously.  Three 
separate  functional  relationships  were  developed,  using  functions 
such  as  power,  active  element  count,  voltage,  and  the  number  of  dis¬ 
play  tubes.  The  first,  (and  only  one  that  includes  the  active  ele¬ 
ment  count)  applicable  to  all  equipments  of  the  type  used  "n  the 
ARINC  study,  is  Indicated  to  be  the  preferred  technique.  The  second 
and  third  equations  are  used  for  radiating  or  non-radiating  systems, 
respectively  The  ARINC  equations  are  given  below  for  similar, 
radiating,  and  non-radiating  equipment,  respectively. 


o  In  0  «  8.6859  -  0.5632Xa  -  0.2556X.  -  0.0858X 

a  d  c 

o  Ln  e  *  6.4408  -  0.2055X.  -  0.1597T  -  0.5120X . 

bed 

o  Ln  e  *  7.2612  -  0.5089X.  -  0.1123Xa 

d  e 


where, 

e  *  Predicted  MTBF  in  hours 

X.  =  Ln  (Adjusted  count  of  active  elements) 

d 

-  Ln  (Power  consumption  in  kw) 

X„  *  N imber  CRT's 
c 

X j  *  Ln  (Max  dc  voltage  in  kv) 

Xj.  *  Ln  (highest  frequency  in  mes) 


The  major  drawback  of  this  technique,  as  with  the  FEC  technique,  is 
that  there  is  a  limited  range  of  electronic  equipment  functions  to 
which  the  technique  can  be  applied  and  the  state  of  the  art  changes 
in  equipment  design  tend  to  make  the  data  obsolete. 

(4)  MIL-STD-756A 

The  MIL-STD-756A  technique  is  a  prediction  by  series  active  elements 
through  the  use  of  a  nomograph.  The  reliability  values  produced  by 
this  method  have  been  shown  (reference  2)  to  be  dependent  upon  the 
functional  block  leyel  to  which  the  equipment  reliability  model  is 
subdivided.  This  dependency  is  illustrated,  for  some  selected  ground 
electronic  equipments,  in  Figure  1  "Ground  Electronic  MTBF  Pre¬ 
dictions  -  First  and  Second  Indenture." 
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Mean  Time  Between  Failure  (Hours) 


The  dashed  line  In  Figure  1  represents  the  predicted  MTBF  to  the  first 
Indenture  level  for_the  selected  equipments  \hile  the  circled  points 
are  the  predicted  MiBF  values  to  the  second  indenture  level.  The  second 
Indenture  level  predictions  assured  a  series  relationship  between  the 
functional  black  boxes,  and  were  plotted  in  Figure  1  using  the  total 
number  of  series  active  elements  in  the  equipment  so  as  to  provide  a 
first  indenture  level  comparison. 


Number  of  Series  Active  Elements 


figure  1,  Ground  Electronics  MTBF  Predictions 
First  and  Second  Indenture 


To  apply  the  prediction  technique,  one  must  first  break  the  system 
up  Into  functional  groups,  count  the  number  of  series  active  elements 
in  each  group,  enter  the  rel lability  nomograph  and  read  off  the  func¬ 
tional  group  MTBF.  The  system  MTBF  is  determined  through  the  system 
reliability  model.  The  active  elements  are  defined  as  tubes,  tran¬ 
sistors,  diodes  and  their  associated  circuitry,  except  that  for  a 
digital  computer  application,  one  tenth  the  number  of  diodes  should 
be  use-*.  An  ao-  tional  Indenture  anomaly  for  an  assembly  which  is  on 
the  border  of  being  classed  a  computer  arises  when  some  of  its  sub- 
assemblies  will  not  be  so  classified.  In  this  case,  a  third  predic¬ 
tion  would  result.  Since  the  MIL-STD-756A  technique  can  be  applied 
for  all  functional  classes  of  electronic  equipment,  this  is  a  tool 
for  providing  a  consistent  set  of  predesign  prediction  techniques, 
provided  predictions  are  made  to  the  same  functional  level. 

c.  Candidate  Equipments 

The  equipments  in  the  predesign  prediction  analysis  are  the  same  equip¬ 
ments  as  were  used  in  Section  IV,  "Improvement  of  Stress  Analysis 
Prediction  Techniques."  The  selection  criteria  used  for  the  stress  analy 
sis  were  also  used,  as  pertaining  to  predesign  predictions  in  this 
section,  and  augmented  to  include  predesign  prediction  requirements.  The 
augmented  list  is  given  below 

o  Availability  of  equipment  parts  lists. 

o  availability  of  predesign  prediction  parameters  on  equipment. 

o  Availability  of  operating  failure  data  on  the  mature  system. 

o  Established  failure  reporting  system  (e.g.,  AF  66-1,  contractor 
reporting,  etc.). 

o  Validity  of  data  for  desired  analysis  techniques. 
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d.  Correlation  With  Observed  Reliability 

Comparisons  of  observed  to  predicted  KTBF's  were  made,  using  predic¬ 
tions  formulated  by  eacn  of  the  pre-design  techniaues.  Since  the 
prediction  by  function  techniques  are  not  applicable  to  the  entire 
class  of  ground  electronic  equipment,  not  all  the  equipment  having 
predictions  formulated  by  the  parts  count  prediction  method  were  used 
in  the  prediction  by  function  analysis.  Furthermore,  it  was  determined 
that  results  having  a  more  general  application  could  be  obtained  from 
the  predesign  prediction  analysis,  if  the  bulk  of  the  analysis  were 
performed  by  the  MIL-ST0-756A  Method,  which  is  applicable  to  the 
entire  class  of  ground  electronic  equipment.  Therefore,  the  ratios  of 
observed  to  predicted  MWs  for  the  ARINC  and  FEC  predictions  are 
shown  for  information  purposes  in  Table  XXV,  "Ratios  of  Observed  to 
Predicted  MTBF  for  Selected  Ground  Electronic  Equipment  -  Prediction 
by  Function." 


TABLE  XXV 

RATIOS  OF  OBSERVED  TO  PREDICTED  MTBF  FOR  SELECTED 
GROUND  ELECTRONIC  EQUIPMENT  -  PREDICTION  BY  FUNCTION 


CODE  NO. 

ARINC 

FEC 

6 

7.06 

21.97 

24 

8.03 

17.09 

33 

.97 

35 

.48 

The  ratios  of  observed  to  predicted  MTBF's  for  MIL-STD-756A  are  shown 
In  Table  XXVI.  The  first  thirteen  equipments  are  in  the  digital 
design  approach  category  while  the  twenty  remaining  equipments  are  in 
the  analog  design  approach  category.  The  MIL-STD-756A  technique,  in 
all  but  one  case,  predicted  MTBF  values  which  are  lower  than  the 
observed,  and  in  some  cases  predicted  considerably  lower  MTBF  values. 
When  the  equipments  were  divided  into  the  two  sub-sets  of  digital 
and  analog,  the  MIL-STD-756A  methoo  exhibited  more  accurate  predic¬ 
tions  for  digital  equipments  than  for  analog. 

Six  functional  relationships  listed  in  paragraph  3. a  above,  of  pre¬ 
dicted  to  observed  MTBF  were  developed  for  each  of  the  following  three 
equipment  categories  and  subcategories: 
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TABLE  XXVI.  RATIOS  OF  OBSERVED  TO  PREDICTED  MTBF  FOR  GROUND 
ELECTRONICS  -  DARTS  COUNT  PREDICTIONS 
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o  All  ground  electronic  equipment 
o  All  analog  electronic  equipment 

-  Power 

-  Controls 

-  Display 

o  All  digital  electronic  equipment 


The  correlation  coefficients  for  each  functional  fit  were  calculated 
to  determine  what,  if  any,  functional  relationships  existed  between 
the  predicted  and  observed  MTBF's.  From  the  correlation  coefficients, 
it  was  determined  that  the  best  fit  of  the  general  categories  was 
provided  by  the  function: 


Where, 

X  *  The  Parts  Count  predicted  HTBF 
Y  *  Adjusted  MTBF 
A  *  Shaping  parameter 
B  *  Shaping  parameter 

This  function,  of  all  the  general  category  functional  relationships  tested, 
provided  correlation  coefficient  values  ranging  from  a  low  of  .608  to  a 
high  of  .867. 


The  correlation  coefficients  for  the  functional  relationship  y  *  AX8 
listed  in  Table  XXVII  for  all  general  categories 


TABLE  XXVII 

CORRELATION  COEFFICIENTS  FOR  THE  FUNCTIONAL 
RELATIONSHIP  Y  =  AXB  -  PREDICTED  TO  OBSERVED  MTBF 


Equipment 

Category 

HIL-STD-756A 

All  Equipment 

.608 

Analog  Equipment 

.622 

Digital  Equipment 

.867 
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For  digital  egulpment.  the  correlation  coefficient  was  not  considered  to 
be  sufficiently  high  to  enable  the  development  of  adjustment  modifiers  to 
modify  MTBF  predictions  of  digital  equipment  to  be  more  In  line  with  the 
observed  HTBF  values.  Further  subdivision  of  the  digital  equipment  was 
not  possible,  so  no  iicroved  prediction  technique  for  digital  equipment 
was  developed. 

for  analog  equipment  the  correlation  coefficient  was  not  as  high  as  for 
the  digital,  however,  analog  equipment  could  be  subdivided  into  three 
njbcetegorles  of  power,  control,  and  display.  For  each  subcategory  the 
six  functions  discussed  In  paragraph  3. a  were  again  fitted.  This  pro¬ 
duced  a  noticeable  Increase  in  the  resultant  correlation  coefficients. 
Table  XXVIII  lists  the  correlation  coefficients,  and  the  shaping  para¬ 
meters  for  those  functions  which  were  found  to  have  sufficiently  high 
correlations.  The  power  and  control  subcategories  had  a  correlation 
coefficient  sufficiently  high  to  use  the  fitted  function  in  adjusting 
predesign  predictions. 


TABLE  XXVI II  SHAPING  PARAMETERS  AND  CORRELATION  COEFFICIENTS 

FOR  SELECTED  PREDICTION  MODIFIERS 
-  ANALOG  EQUIPMENT 


Equipment  Category 
Prediction  Technique 
Fitted  Function 

A 

8 

♦ 

Correlation 

Coefficients 

Power 

MIL-STD-756A 

Y  *  AX8 

1958.0 

.73 

.951 

Control 

MIL-STD-756A 

Y  «  X/(A+BX) 

.0124 

.00037 

.986 

*  “ 

■ 

■  '  " 

48 


SECTION  VI 


INTEGRATED  CIRCUIT  FAILURE  RATES 


1.  OBJECTIVES 

The  objectives  of  this  portion  of  the  study  included  the  following: 

o  Measure  tne  accuracy  of  existing  techniques  for  predicting  the  relia¬ 
bility  of  equipment  using  integrated  circuits  (IC's); 

o  Identify  and  evaluate  major  sources  of  prediction  inaccuracy;  ano, 

o  Establish  failure  rates  and  related  data  useful  for  improving  the 
accuracy  of  readability  predictions  for  equipment  using  IC's. 

2.  SUMMARY 

The  results  of  the  investigation  show  that  IC's  generally  exhibit  failure 
rates  that  are  lower  that  those  predicted  by  two  different  prediction 
models.  The  low  failure  rates  appear  to  be  the  result  of  increasing 
achievement  of  production,  design  and  "Hi-ReV  requirements.  These  same 
factors,  when  applied  to  discrete  parts*  also  result  in  observed  failure 
rates  which  are  lower  than  can  possibly  be  predicted  by  the  most  optimis¬ 
tic  stress  analysis  prediction  technique  of  the  RADC  Notebook  Volume  II. 
The  observed  failure  rates  for  discrete  parts  in  general  show  more  devia¬ 
tion  from  the  predicted  rates  than  do  IC's.  This  large  deviation  from 
predicted  to  observed  still  exists  at  the  equipment  level,  even  though 
parts  such  as  multilayer  printed  circuit  boards  are  not  included  in  the 
prediction  and  contribute  up  to  20%  of  the  observed  failure  rate.  If 
these  parts  were  included  in  the  prediction  there  would  be  a  greater 
difference  between  predicted  and  observed  equipment  failure  rates. 

Prediction  techniques  which  have  limiting  factors,  such  as  the 
modifier  of  the  RADC  Notebook,  prevent  predicted  failure  rates  from 
decreasing  tc  reflect  improvements  in  production,  design,  or  in  the  gen¬ 
eral  state  of  the  art.  The  limiting  factor  for  the  part  failure  rate 
shoiJd  be  the  inherent  failure  rate  of  the  part,  that  is,  the  failure  rate 
associated  with  a  part  which  has  no  quality  defects  and  is  applied  within 
the  design  limits. 

''he  quality  modifier  should  be  a  continuous  type  of  function  instead  of 
establishing  a  few  discrete  points.  This  is  also  brought  out  by  the 
rating  studies  which  show  that  the  ratios  of  predicted  and  observed  equip¬ 
ment  failure  rates  reflect  definite  differences  between  systems  having 
different  quality  programs.  For  the  IC's,  the  Notebook  uses  four  levels 
of  quality,  as  compared  to  two  for  other  parts.  This  may  be  one  reason 
why  the  predictions  for  IC's  are  closer  to  the  observed  than  the  other 
parts. 
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3.  DETAILS 


Review  of  available  literature  shows  that  failure  rates  used  in  relia¬ 
bility  predictions  for  integrated  circuits  (IC's)  can  vary  by  a  factor 
of  100  -  depending  upon  the  manufacturer,  the  environment,  and  the 
user.  This  wide  range  of  variation  in  failure  rates  is  due  to  the 
fact  that  IC's  are  characterized  by  a  very  high  inherent  reliability 
and  a  rapidly  changing  state  of  the  art',  both  of  which  result  in  a 
relative  scarcity  of  standardized  failure  data. 

This  section  presents  an  analysis  of  failure  rate  data  and  evaluations 
of  the  relative  accuracy  of  several  prediction  techniques  applied  to 
a  va*iety  of  IC's.  Failure  rate  information  is  obtained  from  the 
operation  of  these  IC's  in  a  fixed  ground  environment  for  a  total  of 
nearly  eight  billion  hours.  Since  IC's  are  always  used  in  conjunction 
with  discrete  parts,  data  on  the  predicted  and  observed  reliability  of 
these  related  items  a^e  also  included  in  the  analysis. 

a.  Analytical  Techniques  for  IC  Reliability  Prediction 

Table  XXIX  presents  a  summary  of  the  techniques  applicable  to  the  pre¬ 
diction  and  assessment  of  IC  reliability,  which  were  reviewed  during 
this  phase  of  the  study.  The  table  also  presents  conclusions  appli¬ 
cable  to  each  technique. 

Since  there  are  no  pre-design  techniques  applicable  to  the  prediction 
of  IC  reliability  at  this  time,  stress  analysis  techniques  were  used 
In  developing  reliability  predictions  for  analysis  and  comparison  with 
observed  data.  Two  different  prediction  methods  were  selected  as 
follows: 

(1)  The  RADC  Reliability  Notebook,  Volume  II  Method;  and, 

(2)  The  EIA  (Electronics  Industry  Association)  Method. 

The  models  used  by  these  techniques  are  illustrated  in  Table  XXX 
Details  on  the  RADC  method  are  contained  in  the  Notebook  Volume  II. 
Details  of  the  EIA  model  are  discussed  below. 

The  EIA  prediction  model  referred  to  herein  is  the  model  developed  by 
the  EIA  microelectronic  device  group  (MED  4.3).  The  EIA  model  assumes 
the  existence  of  a  mixed  IC  population  comprised  of  two  (2)  major 
subsets.  One  subset  contains  potential  failure  modes  related  to 
inherent  (design-oriented)  factors  which  are  by  definition  non- 
screenable.  The  other  subset  contains  potential  failure  modes  related 
to  quality  assurance  factors  which  are  by  definition  controllable 
and/or  screenable.  Ths  basic  prediction  equation  is  of  the  form: 
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TABLE  XXIX 

ANALYTICAL  TECHNIQUES  FOR  I.C.  RELIABILITY  PREDICTION  AND  ASSESSMENT 
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TABLE  XXIX  (Continued) 

ANALYTICAL  TECHNIQUES  FOR  I.C.  RELIABILITY  PREDICTION  AND  ASSESSMENT  (Concluded) 
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TABLE  XXX 


COMPARISON  OF  FORMATS  FOR  IC  PREDICTION  MODELS 


EIA 

xu  =  ^i  +  prpuV  W^a 

A.  =  +1  --  Design  +  Material 

I  u  is  3 

Failure  Rate  —  Value  .02 
F/H  hours.  Fixed:  or  use  of 
existing  data. 

P£PU'Q  —  Added  Failure  Rates  due 
to  screen  tests  both 
vendor  and  user. 

—  Handling  Factor 

.3  to  1  less  at  cowr.  grade 

K,  —  Tessperature  Factor 

i 

--  Stress  Factor 
Asssmo  to  be  1 

K,  —  Environmental  Factor 
A 

Hot  considered 
Hot  Considered 


RADC  Notebook  Volwe  II 

i  -  l.  X  7r  X  T  X  T-  X  T_ 

#  b  C  p  t  Q 

ijj  —  8ase  Failure  Rate  varies  as 

temperature  froc  .oc«&  to  -035? 
F/Mili  hours . 


Tq  —  Screening  Factor 


Sot  Used 

Covered  by  base  failure  rate 

Not  Used 

I 

Tg  —  Environmental  Factor 
T£  --  Complexity  Factor 
rp  —  Packaging  Factor 


s 


i  >i  +  p£  >Q  ^ 


A 


u 


(S) 


where, 

iy  denotes  the  failure  rate  seen  by  the  user 

aj  denotes  inherent  failures  (non-screenable) 

denotes  QA- related  failures  (control  lable/screer.ible) 

is  the  probability  of  escape  past  the  screen 

denotes  application  (environnental )  factors 

In  theory,  the  tern  i.  is  a  variable  dependent  upon  several  design 
factors,  including  a  factor  for  device  complexity,  In  practice, 
however,  the  design  factors  cannot  be  isolated  and  quantified  to  dis¬ 
tinguish  between  design  variations.  Therefore,  a  constant  *,  failure 
rate  of  -0G2S/109Q  hours  is  used  for  predictions  on  standard1  devices. 
Also  in  theory,  the  terns  ?,  and  are  functions  of  individual  pro¬ 
ducers  idiere  the  QA-relatecTfai lures  are  a  function  of  the  individual 
producer's  process  technologies,  and  the  probability  of  escape  (Pc) 
is  a  function  of  the  effectiveness  of  individual  screening  programs. 

In  practice,  however,  the  available  data  allows  only  for  a  broad 
classification  Wiich  relates  these  factors  to  general  classes  of  pro¬ 
grams  (i.e.,  conaercial,  artlitary,  and  hi-rel).  For  all  except  the 
west  rigid  high-reliability  programs,  the  tern  °c\0  is  saxch  larger 
than  the  tern  *j.  w  ** 

Qce  “failing*  of  the  EIA  aodel  ’s  the  assiraption  of  a  constant  failure 
rate.  In  practice,  studies  have  demonstrated  that  the  IC  system 
failure  rats  initially  decreases,  due  to  continuation  of  screening  at 
the  black  box  level  (in-house  operation  and  checkout),  and  due  to 
expending  of  failures  in  the  £A  related  subset.  In  theory,  the  failure 
rate  then  reaches  a  low  stable  level  and  later  increases  as  devices 
age.  In  practice,  however,  the  available  data  does  not 
the  aging  effect.  A  comparison  of  the  effects  caused  by  the  numeric 
Modifiers  of  the  EIA  and  the  Notebook  Volume  II  is  presented  in 
Table  XXXI, 

b.  Description  of  IC  Data  Sources 

All  IC  data  were  obtained  froa  devices  operating  in  a  fixed  ground 
environment  under  controlled  conditions  of  temperature  and  humidity. 
Ambient  temperature  for  all  devices  was  controlled  within  a  range 
which  was  not  permitted  to  vary  beyond  21 °C  to  37CC.  Humidity  for 
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TABLE  XXX! 


COMPARISON  OF  NUMERIC  MODIFIERS 
FOR  IC  PREDICTION  HOOa 


1.  Screening  Factor  Effect 

Conaercia: 

Military 
Hi -Re! 


2.  Temperature  Factor  Effect 

(?«)  253C 
50®C 
753C 
I25*€ 


3.  Basic  Failure  Rite  Witii 
Temperature  (parts  per 
million  hours) 

25°C 

50°C 

75°C 

125*0 


4.  Environmental  Factor 
Lab. 

Sat.  Orbit 
Gd.  Fixed 
Gd.  Mobile 
Gd.  Portable 
Airborne  Inh. 
Airborne  Unh. 
Sat  Launch 
Hissi le 


For  the  RADC  Notebook,  the  values  are:  Lower  Grade  -  30;  Average  -  15; 
Upper  -  2;  and.  Optimum  -  1. 
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devices  in  an  unsealed  container  was  controlled  with  a  maxima  of  SOX 
AH  at  Z7*C  under  nonal  conditions.  The  remainder  were  operated 
within  a  hermetically  sealed  container. 

These  devices  were  operated  in  the  following  types  of  functional  units 
control  assembly,  amplifier  assembly,  computer,  and  a  signal  data 
converter. 

The  circuits  were  of  the  monolithic  epitaxial  construction.  The 
cases  were  either  Kovar  gold  or  cermnic  1/8*  by  1/2“  flatpadcs  with 
up  to  fourteen  external  leads.  The  seal  on  the  metal  package  was 
weld  and  the  seal  on  the  ceramic  was  glass  frit.  There  were  different 
metal izat ion  interconnects  and  internal  wire  systems  with  a  predomin¬ 
ance  of  gold-gold  and  aluminum-al  (minus.  The  internal  wire  bonds 
for  the  gold-gold  systems  were  “ball  bonds*  and  on  the  aluminue- 
altmrinum  were  “wedge  bonds.”  Stress  conditions  were  controlled  at 
less  tfcar.  505  of  maxi  mm  r>ted  power  and  temperature.  Each  circuit 
had  a  preseal  visual  inspection  and  a  minimum  electrical  bum-in  time 
of  250  hours.  Many  manufacturers  were  represented.  However,  one 
manufacturer  supplied  more  than  fifty  per  cent  of  the  total  parts. 

This  exact  amount  was  impossible  to  determine.  All  circuits  were 
manufactured  under  similar  specifications,  processes,  and  quality 
controls. 

c.  Analysis  of  Data 

Table  XXXII  lists  the  IC’s  used  in  the  equipment  by  device  type.  The 
table  includes  for  each  device:  the  number  of  primary  field  failures, 
accumulated  hours  and  the  best  estimate  failure  rate,  using  the  ratio 
of  total  failures  to  total  hours.  Table  XXXIII  Includes  the  best 
estimate  failure  race  for  each  device  type  by  equipment. 

To  further  analyze  the  failure  rates  in  Table  XXXII  a  distribution  of 
the  failure  modes  was  considered.  This  is  shown  in  Table  XXXIV 
which  indicates  that  73.4*  of  the  failure  »>des  could  not  be  identi¬ 
fied  doe  to  the  fact  that  failure  mode  data  were  not  available.  This 
was  due  to  causes  such  as:  mishandling,  “retested  good  after  removal, 
or  electrical  overstresses.  The  indicated  “handling  problems" 
accounted  for  aporoxiaately  25  per  cent  of  the  uaknown  failures.  This 
suggested  that  handling  errors  may  be  a  major  cause  of  1C  failures, 
i.e.,  good  IC's  may  be  damaged  while  a  failure-  is  being  removed  or 
replaced.  For  those  IC’s  which  were  acceptable  on  retest,  the  possi¬ 
bility  ef  incorrect  removal  action  in  the  field  was  indicated.  If  not 
all  of  the  listed  IC  failures  were  due  to  part  defects  (as  opposed  to 
other  errors)  the  overall  failure  rate  for  IC's  in  this  system  would 
be  much  less  than  indicated. 
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TABLE  XXXI! 

INTEGRATED  CIRCUITS 
FIELD  OPERATION  DATA 


— 

Device  Type 

Number 

of 

Failures 

Total 

Accumulated 

Hours 

Failure  Rate  | 
X/IO3  Hours* 

1) 

Flip  Flop 

41 

507,737,130 

.00452 

2} 

Triple  Nand  Gate 

11 

486,755.545 

.00226 

3} 

Clocked  7  Input  Hand  Gate 

17 

779,175,724 

.00218 

4) 

Clocked  Dual  Hand  Gate 

15 

1,278,504,823 

.00117 

5) 

Input  Network 

3 

45,120,650 

00665 

6) 

Output  Driver 

5 

218,112,541 

.00413 

7) 

Hatrix  Switch 

0 

98,170,438 

.00101 

8} 

Lew  Level  Switch 

6 

147,255,732 

.0C4O7 

5) 

Read  Preamplifier 

3 

79,081,782 

.00379 

io: 

Demodulator  Chopper 

4 

84,535,698 

.00473 

11)  Power  Switch 

20 

338,142,792 

.00621 

12)  Unclccked  Duel  A and  6«te 

28 

2,376,656,832 

.0C118 

13)  Triple  Non-Resistive  Nand  Gate 

1 

38,177,412 

.00262 

14)  Input  Network  Low  Cunent 

4 

183,308,153 

.00218 

15)  One  Shot  Hultivibrator 

0 

15,361,748 

.00511 

ie: 

Level  Detector 

3 

76,354,824 

.00393 

it: 

Write  Switch  No.  2 

0 

170,757.908 

.00059 

18! 

General  Purpose  Amplifier  #1 

7 

292,134,787 

.0024 

!9j 

General  Purpose  Amplifier  #2 

3 

122,713,110 

.00244 

20) 

General  Purpose  Amplifier  #3 

0 

5,453,916 

.0183 

21) 

Driver  Switch 

2 

136,993,966 

.00146 

177 

7,881,505,565 

.00225 

i-  .  - .  .  ■  -  -  - . 

i 

i 

? 


♦Assume  one  failure  if  none 
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TABLE  mm 
INTEGRATED  CIRCUITS 

OBSERVED  FAILURE  RATES  BY  EQUIPMENT  ** 


Device  Type 

Equipment  huMbW  1 

— 

8 

9 

10 

11 

12 

TOTAL 

1 

Flip  Flop 

.00436 

.0122* 

.00667 

.00295 

.00452 

2 

Triple  Kind  Gate 

.00)30 

.0183* 

• 

• 

.00404 

.00226 

3 

Clocked  7  input  Kand  Gate 

-00249 

.0122* 

- 

- 

.00112* 

.00218 

4 

Clocked  Dual  Hand  Gate 

.00101 

.00917 

.00916 

- 

.00137 

.00117 

5 

Input  Network 

.0110 

- 

- 

• 

.00560* 

.00665 

6 

Output  Driver 

.00180 

- 

- 

.0116 

.00413 

7 

Matrix  Switch 

.00102 

- 

- 

• 

• 

.00101 

8 

Low  Level  Switch 

.00407* 

• 

- 

- 

- 

.00407 

9 

Read  Preamplifier 

.00379 

- 

- 

- 

- 

.00379 

10 

Demodulator  Chopper 

- 

- 

.00407 

.00917 

- 

.00473 

11 

Power  Switch 

.0200 

.00136 

.00444 

.00407* 

• 

.00621 

12 

Unclocked  Dual  Hand  Gate 

.00126 

- 

.00367* 

- 

.00082 

.00118 

13 

Triple  Non-Res 1 stive 

Hand  Gate 

.00262 

- 

- 

- 

- 

.00262 

14 

Input  Network  Low  Current 

.00)79 

.0183* 

- 

- 

.00303 

.00218 

15 

One  Shot  Multivibrator 

.00611* 

- 

- 

- 

- 

.0061V 

16 

Level  Detector 

.00393 

- 

- 

- 

- 

.00393 

17 

Write  Switch  Ho.  2 

,000621* 

.0367* 

- 

- 

.0140* 

.00059' 

18 

General  Purpose 

Amplifier  #1 

# 

.000748*. 00579 

00333 

.0033 

.0024 

19 

General  Purpose 

Amplifier  #2 

- 

- 

.00147 

.00367 

- 

.00244 

20 

General  Purpose 

Amplifier  #3 

- 

- 

- 

.0183* 

- 

.0183* 

21 

Driver  Switch 

* 

.00153* 

_ 

,00349 

• 

.007* 

.00146 

*N0  FAILURES 

** 

FAILURE  RATES  ARE  %/ 1000  HOURS 


58 


INTEGRATED 


_ FAILURE  HOPE _ 

Not  Available 
Retest  Good 
Analysis  Pending 
Electrical  Overstress 
Holes  in  Oxide 
Oxide  Defect 
Surface  Contamination 
Non-Herroetic  Seal 
Foreign  Material 
Cracked  Die 
Faulty  Diffusion 
Poor  Adhesive 
Faulty  Bond 

Internal  Lead  Misrouted 
Internal  Lead  Lagging 
Internal  Lead  Damage 
Poor  Mask 
Corrosion 

TOTALS 


*  FAILURE  MODES  FOR  IC  FAILURES  01 
NUMBER  12  NOT  AVAILABLE 


ABLE  XXXIV 

RCUIT  FAILURE  MODES 


TOTAL  NUMBER  PER  EQUIPMENT 

■MrniHb.  a.i  a.ii: 

8 

9 

mam 

IIXI 

WL&m 

53 

C 

5 

1 - 

1 

61 

23 

0 

2 

1 

26 

9 

0 

0 

9 

10 

0 

7 

1 

18 

6 

0 

6 

0 

12 

1 

0 

1 

0 

2 

1 

0 

1 

1 

0 

0 

1 

2 

0 

0 

2 

4 

1 

0 

5 

1 

0 

0 

0 

1 

1 

0 

0 

1 

0 

0 

1 

0 

1 

1 

0 

0 

n 

V 

1 

1 

0 

3 

0 

4 

0 

0 

1 

0 

1 

1 

0 

0 

0 

1 

6 

0 

2 

0 

8 

M 

2 

29 

3 

155  _* 

F  EQUIPMENT 
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The  failure  nodes  which  could  be  attributed  to  the  IC  are  process* 
oriented.  A  list  of  observed  process-oriented  failure  sodes  and 
percentage  distribution  is  giver,  in  the  table  below: 


TABLE  XXXV 

PROCESS -ORI EKTED  FAILURE  MOOES 
(Total  Number  41) 


i  . 

Failure  Mode 

I  of  Total 
Process 
Failures 

Failure  Mode 

....  _  ........ 

2  of  Total 
Process 
Failures 

Oxide  Defects 

34.1 

Poor  Metal  Adhesion 

2.4 

Surface  Contamination 

2.4 

Faulty  Bond 

2.4 

Mon-Hermetic  Seal 

2.4 

Internal  Lead  His routed 

2.4 

Foreign  Material 

4.9 

Internal  Lead  Sagging 

9.8 

Cracked  Die 

12.2 

Internal  Lead  Damage 

2.4 

Faulty  Diffusion 

2.4 

Poor  Mask 

2.4 

Corrosion 

19.5 

The  table  above  indicates  that  quality  control  factors  play  a  very 
Important  role  in  the  failure  rates  of  the  IC's.  At  one  extreme,  the 
high  reliability  IC  has  a  failure  rate  of  about  .0022/1000  hours.  At 
the  other  extreme,  when  all  of  the  unknown  failure  modes  represented 
In  the  first  four  categories  in  Table  XXXIV  and  50  per  cent  of  process 
Q.  C.  errors  are  censored  out,  the  achievable  IC  failure  rate  could 
be  less  than  .00052/1000  hours. 

The  failure  rate  data  presented  in  Table  XXXIII  was  analyzed  to  deter¬ 
mine  if  the  failure  rates  of  individual  devices  were  considered 
statistically  different  from  the  expected  failure  rate  of  0,002252/ 
1000  hours.  The  chi-square  test  was  used  to  determine  statistical 
equivalence.  For  each  device  the  observed  number  of  failures  per 
billion  part  hours  was  compared  to  the  expected  number  of  failures  per 
billion  part  hours  (22,5  failures  per  billion  part  hours)  estimated 
from  the  data.  Oeyices  where  the  failure  expectation  per  cell  was 
less  than  five  failures  were  grouped  so  that  the  assumptions  under¬ 
lying  the  test  were  not  violated.  This  resulted  in  12  cells  for  the 
test. 
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If  the  failure  rates  for  all  devices  were  determined  to  be  statistically 
equivalent,  this  would  be  significant  to  the  validation  of  such  elenents 
of  the  RADC  V'olume  II  prediction  method  as  the  complexity  factor.  It 
would  also  be  significant  from  the  standpoint  of  determining  the  sensi¬ 
tivity  of  IC  failure  rates  to  circuit  application  and  to  equipment 
function. 

The  value  of  chi-square  that  was  obtained  is  146.0,  which,  is  greater 
than  the  value  of  chi-square  in  the  statistical  tables  (10  degrees  of 
freedom,  99%  confidence  level).  Thus,  at  a  confidence  level  exceeding 
99%,  it  is  concluded  that  at  least  one  of  the  devices  in  the  table 
exhibits  a  device  failure  rate  which  is  not  equal  to  .00225%/1000  hours. 

d.  Accuracy  of  IC  Prediction  Models 

When  using  the  two  failure  rate  prediction  methods,  major  considera¬ 
tions  are  utilized;  accuracy  at  the  part  level,  and  the  effect  of  the 
prediction  at  the  equipment  level.  Data  which  were  used  to  verify 
accuracy  at  the  part  level  are  the  accumulated  results  of  Table  XXXII 
Table  XXXYI  presents  a  comparison  of  the  observed  and  predicted  failure 
rates  for  each  device,  based  on  predictions  formulated  by  the  Notebook 
Volume  II  and  £IA  prediction  methods.  To  determine  the  accuracy  of 
the  prediction  methods,  the  following  steps  were  taken: 

(1)  Predictions  were  prepared,  usint  two  junction  temperatures,  30°C 
and  60=C.  These  temperatures  rep  -esent  the  maximum  and  minimum 
junction  temperatures  which  the  devices  experienced  in  actual 
operation.  This  given  range  of  t wipe nature  is  a  necessary 
feature  of  the  prediction,  because  the  data  concerning  junction 
temoerature  was  available  only  as  a  range. 
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(2)  If  the  uoserved  failure  rate  fell  within  the  range  of  failure 
rates  established  by  the  predictions  from  the  two  junction  tem¬ 
peratures,  ’’he  prediction  was  considered  reasonably  accurate  and 
the  error  factor  was  equal  to  zero. 

(3)  If  the  observed  failure  rate  was  greater  than  the  predicted  at 
T.  sax  (underestimate),  the  error  factor  was  formulated  as 
follows: 

■obs/yDred  (T.max) 
j 

(4)  If  the  observed  failure  rate  was  less  than  the  predicted  at  I. 
min.  (overestimate)  the  error  factor  was  formulated  as  follows 
and  identified  in  the  tables  by  a  minus  sign  (-). 

-;.pred  (Timin'/'- obs 

J 

The  analysis  of  the  error  factor  gives  an  insight  into  the  accuracy 
of  the  prediction  models: 

(1)  The  Notebook  Volume  I!  has  seven  values  of  zero:  the  EIA  has 
snree. 

(2)  The  Notebook  Volume  II  overestimates  ten  times  and  underestimates 
four  times.  The  EIA  overestimates  seventeen  zines. 

(3)  The  Notebook  Volume  II  has  an  average  error  factor  for  underesti¬ 
mation  of  1.32  and  -3.73  for  overestimation.  The  EIA-average  for 
overestimation  is  -2.70. 

(4)  The  error  factors  of  the  Notebook  Volume  II 's  prediction  for 
linear  circuits  is  significantly  different  than  those  for  lower 
complexity.  The  EIA  methods  show  no  significant  difference. 

There  are  five  device  types  which  have  large  error  factors  greater 
than  2.0  when  using  the  Notebook  Volume  II  model.  They  are  the  write 
switch,  the  matrix  switch  and  three  linear  amplifiers.  The  reason 
that  the  write  and  matrix  switches  have  lower  failure  rates  than 
predicted  is  not  discernible  from  analysis  of  available  data.  The 
quality  and  complexity  factor*,  could  account  for  inaccuracy  in  the 
predictions  for  these  “Hi-Rel"  parts.  The  reason  for  the  amplifiers’ 
high  error  factors  is  due  to  the  influence  of  the  complexity  factor 
on  the  predicted  values.  With  "Hi-Rel,:  screening,  these  linear 
amplifiers  did  not  show  any  significant  increase  in  failure  rate  over 
tire  digital  type  of  devices.  This  does  not  mean  that  linear  amplifiers 
which  are  manufactured  under  normal  screening  will  exhibit  the  same 
failure  rate  as  digital  devices  which  are  screened  in  the  same  manner. 
The  complexity  modifier  may  need  to  be  further  developed  as  a 
function  of  quality  factors.  Unfortunately,  only  one  quality  level  is 
represented  in  the  data.  If  the  complexity  factor  for  all  devices 


with  “Hi-Sel*  screening  had  been  equal  to  'one*,  then  there  would  be 
only  one  error  factor  over  2.0.  There  would  also  be  an  equal  distri¬ 
bution  between  underestimation  and  over-estimation  of  the  failure  rates. 
However,  when  complexity  reaches  a  medium  scale  integration,  i.e. , 
around  30  gates,  the  effect  of  complexity  on  the  increase  ie  the 
nuaber  of  failure  modes  say  not  be  offset  by  *Hi-Rel*  screens. 

The  EIA  model  has  the  largest  error  factor  and  overestimates  for  90S 
of  the  devices  with  cn=.  underestimation.  The  model  gives  a  range, 

0  to  -4.95,  for  the  error  factors  when  omitting  the  one  large  factor. 
This  is  less  than  the  range  of  1.66  to  -6.78  for  the  Notebook  Volume 
II.  This  indicates  distribution  of  error  is  smaller  for  the  EIA 
model .  However,  with  the  modifiers  and  base  failure  rates,  the  model 
cannot  predict  less  than  .00451  per  !D30  hours  for  these  devices. 

e.  Distribution  of  Equipment  Prediction  Errors 

To  identify  the  sources  of  prediction  error  for  the  equipments  using 
IC's,  the  contribution  of  the  various  parts  to  the  predicted  and 
observed  failure  rates  was  studied.  Tables  -XXYII  through  XLI  present 
the  contribution  of  each  generic  part  typ®  ,o  the  predicted  from  the 
Notebook  «cluoe  II  and  observed  failure  rates  for  the  five  equipments 
in  terrrs  cf  percent  per  1090  fours  and  percent  of  the  total  failure 
rate.  The  Notebook  volume  II  was  used  because  other  prediction 
methods  had  less  accuracy  for  these  equipments.  Observation  of  the 
tables  showed  that: 

(1)  The  prediction  accuracy  for  IC's  and  capacitors  was  better  than 
for  res* stors,  diodes  and  transistors. 

{2}  The  relative  contribution  to  the  observed  failure  rate  of  IC's 
and  capacitors  is  greater  than  predicted,  and  the  relative  con¬ 
tribution  is  subsequently  lower  for  diodes,  transistors,  and 
resistors. 

f 3}  The  influence  of  multilayer  printed  circuit  boards  is  high 

compared  to  the  nucber  of  boards  used.  The  best  estimate  observed 
failure  rate  for  the  awl ti layer  printed  circuit  boards  is  .0585/ 
1005  nc-urs.  The  Notebook  Yo^was  II  does  not,  at  present,  include 
a  fflctr.od  to  sake  a  prediction  for  printed  circuit  boards.  For 
some  systems,  therefore,  the  Notebook  Volume  IJ  wi'l  sot  identify 
a  component  which  is  one  of  the  major  sources  of  failure. 

For  four  of  the  equipments  a  further  breakdown  of  specific  parts 
failures  is  possible.  Tables  XLI I  through  XLV  present  the  total 
accumulated  hours,  number  of  failures,  and  a  best  estimate  failure 
rate  for  specific  parts  by  equipment.  One  failure  is  assumed  if  none 
are  observed.  Table  XLYI  gives  the  best  estimate  failure  rate  for 
specific  parts,  using  the  combined  total  froc  the  previous  four  tables. 


FAILURE  DISTRIBUTION  OF  GENERIC  PARTS  * 
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TABLE  XL 

FAILURE  DISTRIBUTION  OF  GENERIC  PARTS*1 


TABLE  XL I 

FAILURE  DISTRIBUTION  OF  GENERIC  PARTS* 
EQUIPMENT  NUMBER  12 
SIGNAL  DATA  CONVERTER 
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L 


FAILURE 


PART  TYPE 


Capacitor 
tantalum  Foil 
Tantalum  Wet  Slug 
Glass 
Ceramic 

Resistor 

Wi rewound  Accurate 
Film-Precision 
Power  Wi rewound 

Diode 

Silicon  General  Purpose 
Silicon  Power 

Transistor 
Silicon  General  NPN 
Silicon  General  PNP 
Silicon  Power  NPN 
Germanium  General  NPN 
•  Germanium  General  PNP 


*  ASSUME  ONE  FAILURE  IF  NONE 


TABLE  XLIII 

FAILURE  DISTRIBUTION  OF  DISCRETE  PARTS 
EQUIPMENT  NUMBER  9 
AMPLIFIER  ASSEMBLY 


PART  TYPE 

NUMBER 

OF 

FAILURES 

TOTAL 

ACCUMULATED 

HOURS 

BEST  ESTIMATE 
FAILURE  RATE 

%/103  HOURS* 

Capacitor 

Tantalus  Foil 

0 

19,088,706 

.0052 

Tantalus  Wet  Slug 

29 

46,358,286 

.0626 

Ceramic 

0 

50,994,146 

.00196 

Polystyrene 

] 

19,088,706 

.0C524 

Resistor 

VI i  rewound  Accurate 

0 

730,824,744 

.000136 

Precision  Film 

0 

1,783,430,532 

.0000561 

Power  Wi rewound 

0 

398,135,868 

.000251 

Diode 

Silicon  General 

1 

1,308,939,840 

.0000764 

Silicon  Power 

0 

57,266,118 

.00175 

Transistor 

Silicon  General  NPN 

1 

302,692,338 

.000330 

Silicon  Genera!  PNP 

0 

16,361,748 

.00611 

Silicon  Power  NPN 

1 

_ — 

139,074,858 

.000719 

*  ASSUME  ONE  FAILURE  IF  NONE 


TABLE  XLIV 


FAILURE  DISTRIBUTION  OF  DISCRETE  PARTS 
EQUIPMENT  NUMBER  10 
CONTROL  UNIT 


PART  TYPE 

NUMBER 

OF 

FAILURES 

TOTAL 

ACCUMULATED 

HOURS 

BEST  ESTIMATE! 
FAILURE  RATE  I 

%/103  HOURS* 

Capacitor 

Paper 

0 

19,088,706 

.00524 

Tantalum  Wet  Slug 

24 

886,261,350 

.00271 

Glass 

0 

62,720,034 

.00159 

Ceramic 

1 

662,650,794 

.000151 

Polystyrene 

0 

5,453,916 

.0183 

Resistor 

Wi rewound 

1 

13,634,790 

.00733 

Precision  Film 

0 

1,079,875,368 

.0000926 

Power  Wi rewound 

0 

474,490,692 

.000211 

Diode 

Silicon  General 

18 

2,094,303,744 

.000859 

Silicon  Power 

0 

152,709,648 

.000655 

Transistor 

Silicon  General  NPN 

6 

670,831,668 

.000894 

Silicon  General  PNP 

10 

286,330,590 

.00349 

Silicon  Power  NPN 

4 

84,535,698 

.00473 

♦ASSUME  ONE  FAILURE  IF  NONE 


73 


TABLE  XLV 


FAILURE  OISTPIBUTIOH  OF  DISCRETE  PARTS 
EQUIPMENT  NUMBER  11 
ACCESSORY  ELECTRONICS 


PART  TYPE 

NUMBER 

OF 

FAILURES 

TOTAL 

ACCUMULATED 

HOURS 

BEST  ESTIMATE 
FAILURE  RATE 

%/TO3  HOURS* 

Capaci tor 

Tantalus  Foil 

0 

81,808,740 

.00122 

Tantalus  Met  Slug 

3 

122,713,110 

.00244 

Glass 

0 

10,907,832 

.00917 

Ceramic 

0 

234,518,388 

.000426 

Polystyrene 

0 

8,180,874 

.0122 

Resistor 

Wi rewound  Accurate 

0 

19,088,706 

.00524 

Precision  Film 

0 

286,330,590 

.000349 

Power  Wi rewound 

0 

109,078,320 

.000917 

Diode 

Silicon  General 

0 

158,163,564 

•  OOOcji. 

Transistor 

Silicon  General  NPN 

1 

111,895,278 

-.000894 

Silicon  General  FNP 

1 

79,081,782 

.00126 

*  ASSUME  ONE  FAILURE  IF  NONE 
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PART  TYPE 

||9|| 

TOTAL 

ACCUMULATED 

HOURS 

Capacitor 

Paper 

0 

19,088,706 

Tantalum  Foil 

0 

141,801,816 

Tantalum  Wet  Slug 

72 

1,709,802,666 

Glass 

0 

117,259,194 

Cerami c 

2 

1,343,572,238 

Polystyrene 

*1 

l 

32,723,496 

Resistor 

Wi rewound  Accurate 

3 

1,112,598,864 

Precision  Film 

0 

6,861,026,328 

Power  Wi rewound 

0 

2,546,978,772 

Diode 

Silicon  General  Purpose 

39 

6,610,146,152 

Silicon  Power 

1 

619,019,466 

Transistor 

Silicon  General  NPN 

•E 

L  - 

1, SCI, 246, 196 

Si 1 1 r  on  General  PNP 

15 

788,090,862 

Silicon  Power  NPN 

11 

278,149,716 

Germanium  General  NPN 

0 

106,351,362 

Germanium  General  PNP 

2 

95,443,530 

BEST  ESTIMATE, 
FAILURE  RATE 

%/103  HOURS* 


.00524 

.000705 

.00421 

.000G53 

.000149 

.00306 


.000270 
.0000146 
. 0000393 


.000590 

.000162 


.00138 

.00190 

.00396 

.000939 

.00210 


* ASSUME  ONE  FAILURE  IF  NONE 
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Mhen  comparing  the  observed  part  failure  rates  with  the  Notebook 
Voliae  II  predictions,  these  "hi-Rel"  parts  generally  experienced 
lower  failure  rates  than  the  "upper  quality  grade"  parts,  except  for 
the  tantalum  wet  slug  capacitor  and  those  parts  without  enough 
accumulated  hours.  Table  XLVIII  compares  the  observed  failure  rate 
with  the  Notebook  Volume  II  predictions.  The  table  shows  that: 


(1)  Almost  all  parts  have  a  lower  observed  than  predicted  failure 
rate. 

(2)  The  germanium  NPN  transistor  failure  rate  was  less  than  that  for 
silicon  transistors.  This  is  contrary  to  the  generally  accepted 
belief  that  silicon  transistors  exhibit  failure  rates  lower  than 
germanium.  The  lower  failure  rates  arc  believed  to  be  due  to  the 
effects  of  “hi-Rel”  screens  and  low  power  dissipation  (less 

than  501  of  maximum  rated  power  capabilities). 

(3)  The  tantalum  wet  slug  capacitor  is  the  only  type  of  part  which 
has  a  higher  observed  failure  rate  than  the  predicted.  The  hi qh 
failure  rate  is  due  to  an  application  problem. 

(4)  The  observed  resistor  and  diode  failure  rates  are  less  than  the 
upper  grade  additive  failure  rate  Ur)  derived  from  the  Noteoouk 
Volume  II  formula  {xpart  =  xbase  X  it  modifiers  +  lr) .  The  z £ 
factor,  determined  by  environment  and  part  quality  grade,  becomes 
the  limiting  factor  for  the  failure  rate  of  these  parts.  In 
other  words,  the  predicted  part  failure  rate  cannot  be  less  than 
the  factor.  Examples  are  shown  in  Table  XLVII  below. 


TABLE  XLVII 

COMPARISON  OF  OBSERVED  FAILURE 
RATES  AND  Z£  FACTOR  VALUES 


Part  Type 

Failure  Rate  %/l 000  Hours 

Observed 

T  * 

Resistor 

Accurate  Wi rewound 

.00027 

.002 

Precision  Film 

.0000146 

.001 

Power  Wi rewound 

.0000393 

.0015 

Diodes 

Silicon  General  Purpose 

.00059 

.001 

Silicon  Power 

.000162 

.001 

*  Derived  from  RADC  Notebook  Volume  II  using  fixed 
ground  environment  and  upper  grade  quality 
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ItSLE  XLY1II 


COfrAfclSQS  Of 


r 

! 


WQ>  PREDICTED  FAILURE  RETES* 
FOR  DISCRETE  PARTS 


— 

Part  Type 

' 

' 

1  Gfcserred 

1  Predicted 

7  Observed 

CAPACITOR 

Paper 

.00524 

.00005 

Tantalum  Fell 

.0GC7G5 

.005 

7.09 

Tantalum  iiet  Slue 

.00421 

.002 

.475 

Glass 

.000353 

.0001 

*** 

Ceramic 

.000149 

.0007 

4.70 

Polys lyrene 

.03306 

.058 

2.51 

RESISTOR 

Accurate  if:  rewound 

.000270 

.037 

25.9 

Precision  Fils 

.0000146 

.0005 

34.2 

Power  Sii  rewound 

.0000393 

.0054 

138 

| 

DIODE 

Silicon  General  Purpose 

.00959C 

.035 

8.47 

Silicon  Power 

.000162 

.007 

43.2 

TRAHSI5T0R 

Silicon  General  MP8 

.00138 

.013 

9.42 

Silicon  General  PKP 

.00190 

.017 

8.95 

Silicon  Power  Jf?K 

.00396 

.025 

6.31 

Ge  General  NPM 

.QG0939 

.098 

104.4 

Ge  General’  PUP 

.(K3210 

.065 

31.0 

sEcnm  vii 


SUABILITY  D0GSS77.ATIOR  APPROACH 


1.  OCJECTIYcS 

This  sectice  presents  a  comparison  of  four  general  approaches  to  reliabil¬ 
ity  demonstration  for  groead  electronic  equipment-  as  follows: 

o  For**]  Sell  ability  Swore  tret!  on 

o  Field  Reliability  Ceaeastratioe 

o  Reliability  Verification  Testisg 

o  Reliability  Demonstration  by  Prediction 

The  adratages  and  1  in;  tat  ices  of  esds  approach  are  listed,  and  based  on 
these  comparisons,  a  suggested  reliability  deRcasxratiea  approach,  incor¬ 
porating  features  cf  the  above  approaches,  is  formulated. 

Also  iaciaded  are  results  cf  statics  and  analysis  relating  to  the  Question 
of  reliability  democstrati  cr=.  Specifically,  tae  results  of  Special  Study 
Mo.  5  and  Special  Study  Sto.  1  cf  Section  ’fill  ere  included,  as  these 
studies  bear  directly  on  the  question  of  reliability  danoascratie©. 
Finally,  a  discussion  of  Bayes  Theorem  is  included,  describing  a  methods!  - 
ogy  idscreby  prior  subjective  reliability  estimates  can  be  merged  with  test 
data  to  obtain  reliability  estimates  of  assemblies  and  equipments. 

2.  answer 

The  suggested  reliability  demonstration  approach  consists  of  three  parts, 
as  follows: 

{!)  Ifiltially  employ  reliability  predictions  tn  fora  a?3  equipment 
reliability  baseline. 

(?)  Develop  and  implement  a  reliability  verification  test  program 
designed  to  assure  that  failure  mechanises  are  identified  and 
corrective  action  is  implemented. 

(3)  Perforas  a  formal,  in-service  reliability  demonstration  test  to 
verify  equipment  reliability  under  use  conditions.  Apply  a  formal 
accept/reject  criteria  to  the  test. 

This  suggested  approach  to  reliability  demonstration  is  designed  to  gain 
maxis*  assurance  that  the  equipment  does  possess  a  reliability  level 
idiich  meets  (or  exceeds)  developed  equipment  reliability  requirements. 
However,  in  performing  a  forsal  reliability  demonstration  test,  care 
Bust  be  exercised  to  assure  that  the  basic  statistical  assumptions  em¬ 
ployed  in  formulating  the  test  are  not  violated,  specifically,  the 
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assumption  of  a  constant  failure  rate.  Special  Study  Ko.  5,  and  Special 
Study  So.  T  of  Section  VIII  indicate  that  the  assucption  of  constant  fail¬ 
ure  rate  snay  not  be  generally  applicable,  and  that  reliability  prediction 
techniques  cay  not  be  sufficiently  accurate  such  that  meaningful  demonstra¬ 
tion  requirements  can  be  established,  based  on  these  predictions. 

Considering  the  equipspent  represented  in  Table  LXXYI,  ‘Percentage  Failure 
Rate  Change”  (Special  Study  -Vo.  5,  Section  VIII),  51.75  had  a  decreasing 
failure  rate  vs  time,  34.55  had  a  constant  failure  rate  vs  tiiae,  and  13.85 
bad  an  increasing  failure  rate  vs  time.  Thus,  65.55  of  the  equipnent  had 
failure  rate  vs  time  characteristics  which  were  in  variance  with  the 
statistical  model  assumed.  Decisions  on  whether  the  equipment  passed  or 
failed  the  test  say  not  necessarily  be  valid.  For  example,  it  is  possible 
that  an  equipment  exhibiting  an  increasing  failure  rate  vs  tier  could  pass 
a  fixed  time  test  simply  because,  during  the  test,  tfee  total  number  of 
failures  observed  did  rot  exceed  the  allowable  number  of  failures  defined 
by  the  statistics  of  the  test.  Equipment  operation  for  a  slightly  longer 
period  of  time  may  alter  the  ccsxlusion.  Likewise,  an  equipnent  Which 
exhibits  a  decreasing  failure  rate  vs  time  may  fail  the  demonstration  test, 
but  nay,  in  fact,  nave  an  adequate  failure  rate,  as  nay  nave  been  demon¬ 
strated  if  the  test  bad  not  been  terminated.  Only  if  the  equipment  failure 
rate  is  actually  constant  can  valid  conclusions  be  dram  concerning  the 
demonstrated  failure  rate  and  the  validity  of  tfee  demonstration  test. 

A  second  inference  that  can  be  drawn  from  the  results  of  Special  Study  So. 

5,  Section  VIII,  is  that  decreasing  failure  rates  vs  time  can  be  expected 
in,  perhaps,  up  to  505  of  tfee  equipments.  The  implication  is  that ,  under 
the  exponential  failure  assumptions  these  equipments  would  pass  the  test. 
This  be  true  in  genera,  if  tfee  failure  rate  calculated  during  the 

early  part  of  the  test  equaled  tfee  expected  value  of  the  failure  rate. 

The  demonstrated  failure  rate  would  then  be  greater  than  the  actual  failure 
rate  of  tfee  equipment  since  the  failure  rate  would  be  decreasing.  Thus, 
reliability  demonstration  tests  would  yield  pessimist* c  estimates  of  the 
reliability  of  the  equipment.  If  tfee  failure  rate  were  increasing  with 
time,  an  optimistic  estimate  of  tfee  failure  rate  would  result. 

A  third  inference  may  be  drawn  from  tfee  results  of  Special  Study  Ko.  1, 
Section  VIII.  When  early  reliability  predict! ons  are  used  as  the  basis 
for  establishing  quantitative  reliability  demonstration  requi resents ,  the 
probability  and  range  of  possible  error  should  be  considered.  Table  LI¥  , 
“Comparison  of  Predicted  and  Observed  Failure  Rates3  (Special  Study  Ko.  1, 
Section  VIII),  shows  that  84.65  c*f  the  observed  failure  rates  differed 
from  the  predicted  failure  rates  by  greater  than  plus  or  minus  1005.  In 
S5.4S  of  the  cases,  the  predicted  failure  rate  over-estimates  the  observed 
failure  rate  by  a  factor  exceeding  two.  Therefore,  in  these  cases  it 
would  be  expected  that  the  equipment  would  easily  pass  the  reliability 
demonstration. 

Statistical  reliability  demonstration  techniques  would  not  be  appropriate 
for  most  of  these  equipments  because  reliability  daaonstration  models  assume 
constant  failure  rates  and,  72.95  uf  the  equipment  exhibit  failure  rates 
which  are  not  constant  with  time  (See  Table  LXXY,  "Equipment  Failure  Rate 
Characteristics  vs  Time,1’  Special  Study  No.  5,  Section  YI 1 1). 
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3.  DETAILS 


t.  Introduction 

This  section  of  the  study  presents  a  brief  review  of  the  current  deson- 
st ration  approaches,  their  advantages  and  limitations,  and  a  recossended 
approach.  For  the  purposes  of  this  discussion,  reliability  deaonstration 
is  viewed  as  a  aeans  of  obtaining  assurance  that  the  equipment  will  per¬ 
form  the  required  functions  in  the  specified  environment  without  failure 
for  a  specified  period  of  tiae.  Current  approaches  to  reliability  demon¬ 
stration,  unoe r  this  definition,  include: 

o  Formal  test  of  the  equipment  for  the  purpose  of  determining  whether 
the  equipment  sects  a  specified  reliability  requirement.  The  test 
results  are  analyzed  employing  statistical  analysis  techniques. 

This  is  generally  accomplished  3t  the  contractor' s  facility. 

o  In-service  equipment  dessors trailer,  i?.  the  field  once  the  equipment 
has  been  installed  and  debugged.  This  type  of  testing  results  in 
an  estimation  of  "use*  reliability.  Generally,  the  test  results 
are  analyzed,  employing  statistical  techniques,  but  the  test  say 
l«ck  the  formality  of  the  forsal  dasonstration  test.  The  stasis 
is  placed  upon  measuring  the  achieved  equipment  reliability  rather 
than  formally  demonstrating  a  specified  reliability  requirement, 
although  field  testing  has  been  employed  for  formal  deaonstration 
Gf  equipment  reliability. 

o  Reliability  verification  tests,  designed  to  gain  assurance  that 
the  equipment  will  operate  as  specified  in  a  reliable  manner. 

This  type  of  testing  includes  qualification  or  environmental 
testing.  This  type  cf  testing  does  not  result  in  a  formal 
assessment  (in  the  statistical  sense)  of  equipment  reliability. 

o  Reliability  demonstration  by  prediction.  The  reliability  predic¬ 
tion  is  accepted  as  the  demonstration  cf  equipment  reliability. 

Each  of  these  demonstration  techniques  is  discussed  in  more  detail  in  the 
following  sections.  The  features  of  each  type  of  test,  the  advantages, 
and  the  1 isolations  of  each  approach  are  presented. 

b.  Formal  Deaonstration  Tests 

The  foreal  reliability  demonstration  test  deterainps  whether  the  equipment 
meets  a  specified  reliability  requirement  by  accisulating  operatino  time 
and  failure  data  on  one  or  ©ore  equipments  under  specified  environmental 
and  operational  conditions  in  accordance  with  a  statistical  test  plan. 

The  results  of  the  test  statistically  determine  whether,  within  stated 
confidence  limits,  the  required  reliability  was  demonstrated.  The  under- 
«ying  equipment  failure  Bode!  generally  assisoed  is  the  exponential  time  to 
failure  d'.stribution  and  the  accept/ reject  criteria  are  ba^ed  upon  the" 


Poisson  distribution.  The  daaonstration  test  is  usually  conducted  at  the 
contractor's  facility  by  contractor  personnel  under  the  cognizance  of 
cus toner  representatives. 

The  demonstration  test  is  a  fomai  procedure  characterized  by  strict  opera¬ 
tional  rules,  rigidly  defined  result  stateaents,  considerable  expense,  and 
long  .'■est  times  neasured  in  multiples  of  tb*  required  HTBF.  The  demon¬ 
stration  test  plan  consists  of  four  parts: 


c  Procedure  rules 
o  Decision  rules 
o  The  experimental  outcome 
o  Stateaents  of  results 

The  procedure  rules  are  a  priori  statements  of  the  test  procedure,  and 
include: 

o  The  type  of  test;  fixed  time,  fixed  r.uefcer  of  failures,  sequential 
or  other  applicable  testing  approaches. 

o  The  specific  decision  rules,  accept/reject  criteria,  and  definitions 
of  failure  and  degradation. 

o  The  number  o*  equipments  to  be  employed  during  the  test  and  the 
test  time  for  each  equipment. 

o  Rules  under  which  procedures  for  unforeseen  circunstances  or  questions 
will  be  determined. 

o  The  penalties  if  the  equipment  fails  the  test  and  the  rewards  if  the 
equipment  passes  the  test. 

The  decision  rules  depend  upon  the  type  of  test  and  the  specific  relia¬ 
bility  specification  which  is  applied  to  the  program,  but  in  general  will 
be  composed  of  an  accept/ reject  criteria  based  upon  the  appropriate 
statistics  and  statistical  distribution  assumptions  inherent  in  establish¬ 
ing  the  test.  The  decision  rule  will  associate  the  nuir.  er  of  failures, 
test  times,  penalties  and  rewards  with  the  decision  to  accept/ reject. 

Tne  experimental  outcome  is  the  set  of  data  which  becomes  available  as  a 
result  of  the  test.  These  data  are  analyzed,  using  the  decision  rules  and 
employing  suitable  statistical  analysis" techniques.  Determination  is  made 
as  to  whether  the  equipment  has  passed  or  failed  the  test.  This  determi¬ 
nation  is  the  statement  of  the  results  of  the  test,  and  once  the  results 
are  verified,  the  test  is  completed.  Further  courses  of  action  are  then 
pursued,  based  upon  the  procedure  and  decision  rules  established  prior  to 
the  test.  Advantages  and  limitations  are  as  follows: 
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-ADVANTAGES” 


o  Provides  a  formal  demonstration  of  equipment  reliability  at  a  stated 
confidence  level. 

o  Provides  definite  penalties  and  rewards,  based  upon  the  results  of 
the  demonstration  test,  which  act  as  powerful  incentives  to  reli¬ 
ability  achievement. 

o  Permits  early  assessment  of  reliability  which  allows  the 
identification  and  correction  of  problems  before  equipment  is 
shipped  to  the  field. 


-LIMITATIONS- 

o  The  cost,  test  time,  and  number  of  equipments  required  to  demonstrate 
statistically  a  required  reliability  may  be  excessive.. 

o  Since  the  formal  demonstration  cannot  fully  duplicate  or  simulate  the 
actual  field  environment,  the  results  are  questionable. 

o  The  statistical  assumptions  used  to  formulate  the  test  may  not  be 
valid;  e.g.,  the  exponential  failure  distribution  model  is  not 
applicable.  (See  Special  Study  No.  5,  Section  VIII) 

o  The  test  articles  often  are  not  representative  of  the  final  product, 
due  to  changes  in  production  processes,  design,  and  materials. 

o  Difficulty  is  encountered  determining  courses  of  action  under  un¬ 
foreseen  circumstances,  even  though  provision  is  made  for  these  type 
occurrences;  l.e.,  all  failure  and  degradation  modes  cannot  be  de¬ 
fined  In  advance. 

o  It  may  not  be  feasible  to  perform  a  formal  demonstration  test,  due 
to  the  size  of  the  equipment  or  the  operations  which  are  required 
to  be  simulated. 

c.  Field  Reliability  Demonstration 

The  field  type  of  reliability  demonstration  has  many  features  which  are 
similarto  the  formal  demonstration  test.  The  basic  test  also  accumulates 
operating  hours  and  failure  data  on  the  equipment  in  accordance  with  a 
statistical  test  plan.  Statistical  techniques  are  employed  to  analyze 
the  data.  The  underlying  failure  model  that  is  assumed  is  exponential. 
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Procedure  rules  and  decision  rules  are  formulated  for  the  test. 

Generally,  contractor  personnel  perform  the  test  under  the  cognizance 
of  customer  personnel. 

However,  despite  these  similarities  there  are  substantial  differences  in 
the  field  demonstration  test  and  the  formal  demonstration  test.  The 
major  difference  is  that  the  equipment  is  tested  in  the  actual  operating 
environment.  The  actual  stress  and  environmental  conditions  that  will  be 
present  during  the  life  of  the  equipment  are  present  during  the  demon¬ 
stration  test. 

A  second  major  difference  is  that  the  effects,  on  equipment  reliability, 
of  the  in-service  operational  and  maintenance  policies  and  procedures 
will  influece  the  test  results.  For  example,  improper  operational  or 
maintenance  practices  could  be  reflected  by  an  increased  equipment  fail¬ 
ure  rate.  Also,  the  effects  of  logistics  support  policies  and  diagnostic 
techniques  can  be  significant  to  the  results  of  this  test.  (These 
policies  and  techniques  will  influence  the  mean  time  to  repair  (HTTR)  of 
the  eauipment  rather  than  the  mean  time  td  failure.  On  ground  electronic 
equipment,  the  MTTR  is  a  significant  parameter,  contributing  to  system 
availability. ) 

A  third  major  difference  that  generally  exists  between  the  two  testing 
concepts  is  that  a  formal  accept/reject  criteria,  including  appropriate 
penalties  and  rewards,  is  not  established.  Instead,  the  test  is  run  to 
determine  the  level  of  reliability  that  is  achieved  and  to  pinpoint  pro¬ 
blem  areas  that  require  corrective  action.  The  test  then  becomes  an 
assurance/corrective  action  test  rather  than  an  accept/reject  test.  If 
the  equipment  reliability  demonstrated  is  below  a  desired  or  required 
level,  the  only  recourse  available  to  the  customer  is  to  require  that 
corrective  action  be  implemented  on  those  areas  where  a  high  defect  rate 
was  observed.  But,  on  the  other  hand,  if  the  equipment  performs  up  to 
or  exceeding  expectations,  the  customer  has  gained  assurance  that  the 
equipment  performs  reliably  in  the  actual  operating  environment.  This 
assurance  is  gained  without  the  requirement  to  pay  an  incentive  fee. 

Recently,  field  reliability  demonstration  tests  with  definite  accept/ 
reject  criteria  and  associated  penalty  and  rewards,  have  been  proposed, 
and  at  least  one  such  test  is  in  progress  today.  This  test  requires  the 
demonstration  of  a  specified  minimum  MTBF.  If  the  MTBF  is  not  achieved, 
the  contractor  will  incur  a  financial  penalty.  If  the  MTBF  is  achieved 
or  exceeded,  the  contractor  will  gain  an  incentive  fee.  Thus,  this  in- 
service  test  is  an  in-service,  formal  demonstration  test  of  the  equipment. 
Advantages  and  limitations  are  listed  below. 


-ADVANTAGES - 

o  The  test  eliminates  the  cost  and  inaccuracies  which  result,  when  the 
environment  is  simulated,  since  it  is  conducted  under  the  actual 
operating  environment  of  the  equipment. 
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o  The  test  cost  Is  minimized  by  the  use  of  existing  operating  per¬ 
sonnel  and  facilities,  In  lieu  of  requiring  special  test  personnel 
and  facilities  solely  for  formal  demonstration  purposes. 

o  Earlier  delivery  schedules  are  achieved  since  no  additional  calendar 
time  need  be  expended  after  production  acceptance  tests  for  the  pur¬ 
pose  of  reliability  demonstration. 

o  The  test  may  identify  significant  areas  for  the  improvement  of  the 
effectiveness  of  the  operational,  maintenance,  and  logistic  areas. 


-UMItATICHS- 

o  Field  operational  and  maintenance  policies  and  practices  are  beyond 
the  control  of  the  contractor  and  the  customer.  This  can  totally 
invalidate  the  test  results. 

o  If  equipment  falls,  it  may  be  too  late  to  Incorporate  the  necessary 
changes  to  correct  the  fault,  for  economic  or  tactical  reasons. 

o  It  is  sla^ly  not  practical  or  feasible  to  monitor  and  measure  the 
reliability  of  many  types  or  equipments  in  the  field  (e.g.,  because 
of  short  duty  cycles,  inaccessible  points  of  use,  delays  in  field 
deployment  schedules,  etc.). 

d.  Reliability  Verification  Tests 

Reliability  verification  testing  represents  a  substantial  departure  from 
the  formal  or  fieTd  approaches  to  demonstration  testing.  In  this 
approach,  the  emphasis  is  not  on  demonstrating  a  specific  reliability 
but  to  verify  that  there  are  no  major  sources  of  unreliability.  Tests 
are  conducted  both  at  the  complete  equipment  level  and  the  level  of 
individual  parts  or  assemblies.  The  general  procedure  is  to  operate  the 
equipment  at  stress  levels  in  excess  of  operating  levels  to  induce  de¬ 
gradation  in  operating  parameters,  identify  significant  failure  modes, 
and  pinpoint  design  features  requiring  optimization  for  the  improvement 
of  reliability  and/or  operating  characteristics. 

The  equipments  to  be  tested,  using  the  verification  testing  technique, 
are  selected  by  employing  one  of  the  following  criteria: 

o  High  risk  designs,  either  state-of-the-art  assemblies  or  assemblies 
which  have  a  past  history  of  a  high  failure  rate. 

o  Items  which  possess  known  failure  or  wearout  mechanisms^  batteries, 
tube  design,  etc. 


84 


o  Items  which,  based  on  a  failure  modes  and  effects  analysis,  have 
unresolved  potential  failure  mechanisms  which  require  investigation. 

A  sample  of  each  of  these  equipments  are  placed  on  test  for  a  period 
sufficient  to  verify  the  existence  or  non-existence  of  the  suspected 
failure  mechanisms.  During  the  test,  special  subtests,  such  as  tempera¬ 
ture  cycling,  vibration,  and  shock  would  be  performed  to  simulate 
assembly  stress  under  actual  use  conditions  and  to  perhaps  accelerate  the 
suspected  failure  mechanism.  The  test  would  be  concluded  when  sufficient 
data  was  obtained  to  verify  the  failure  mechanism  or  to  adequately  de¬ 
monstrate  that  the  assembly  was  suitable  (from  a  reliability  and 
performance  standpoint)  to  incorporate  on  the  final  equipment.  No 
attempt  is  made,  generally,  to  test  a  large  sample  for  a  long  test  time, 
such  that  sufficient  data  suitable  for  statistical  analysis  would  be 
obtained.  However,  if  the  verification  test  required  that  a  large 
sample  be  tested  for  a  substantial  period,  statistical  data  analysis 
would  be  possible.  The  analysis  technique  would  not  require  the  assinp- 
tion  of  exponential  time  to  failure,  particularly  when  a  wearout 
mechanism  was  under  investigation. 

-ADVANTAGES- 

o  Relatively  inexpensive  to  integrate  within  the  standard  qualification 
test  program.  ■* 

o  Less  time-consuming  than  either  of  the  previously  described 
approaches. 

o  Designed  to  pinpoint  failure  mechanisms  and  design  inadequacies 
for  early  corrective  action.  The  emphasis  is  placed  upon  elimina¬ 
tion  of  potential  failure  mechanisms  and  upon  understanding  of  other 
failure  causes  (such  as  wearout). 

o  When  statistical  data  analysis  is  possible  it  allows  freedom  in 
selection  of  the  most  appropriate  failure  distribution  which  more 
accurately  describes  phenomena  under  study. 

o  Aimed  at  building  reliability  into  the  equipment.  Other  testing 
approaches  only  identify  weak  areas  in  the  equipment.  This 
approach  requires  corrective  action  to  be  incorporated  in  the 
design. 

-LIMITATIONS- 

o  Requires  considerable  engineering  manhours,  scheduling 
and  funding  to  support  program. 

o  Provides  no  accept/ reject  criteria. 

o  Does  not  test  the  complete  equipment  for  a  period  of  time 
sufficient  to  demonstrate  reliability. 


o  Does  not  sireulata  the  actual  operating  environment, 
e.  Demonstration  3y  Prediction 

Equipment  demonstration  by  prediction  siaply  aeans  that  the  reliability 
prediction  is  accepted  as  verification  that  the  equipment  possesses  a 
satisfactory  reliability  level.  This  technique  does  not  require  any 
testing,  either  at  the  equipment  or  assembly  level ,  but  may  use  data 
obtained  fro*  such  testing  to  validate  predictions. 

The  procedure  for  performing  this  type  of  dawns trati on  is  tc  toisulate 
a  reliability  prediction  in  the  accepted  Banner.  The  contractor  develops 
a  Mathematical  Model  representing  the  reliability  characters  sties  of  the 
equipment.  The  sodel  divides  the  equipment  into  assemblies  or  sub¬ 
assemblies.  Next  a  reliability  estimate  is  generated  for  each  of  the 
assemblies  and  subassemblies,  using  any  one  of  several  acceptable  pre¬ 
diction  techniques.  These  include  stress  analysis  prediction  techniques, 
prediction  by  siHilarity  or  other  appropriate  Methods.  The  reliability 
of  each  of  the  assemblies  is  combined  by  use  of  the  appropriate  Mathemati¬ 
cal  expressions  and  an  estimate  of  the  equipment  reliability  is  generated. 
The  entire  process  Is  Monitored  and  subject  to  approval  by  the  customer 
to  Insure  adequate  consideration  is  given  to  all  factors  which  influence 
the  equipment  reliability.  The  applicability  and  acceptability  of  fail¬ 
ure  rate  data  incorporated  in  the  prediction  is  the  key  issue. 

-ADYANTAGES- 

o  Major  program  cost  reductions  are  achieved  by  eliminating  test 
requirements. 

o  lamed! ate  corrective  action  response  capability  is  achieved, 

o  Frograa  delivery  schedules  are  shortened. 

-IIHITATIONS- 

o  Established  prediction  inaccuracies  introduce  large  risk  factors. 

o  Accept/reject  criteria  for  the  prediction  procedure  are  difficult 

to  establish  and  may  require  extensive  negotiation. 

o  The  effects  of  the  field  environments-including  operational  and 
maintenance  factors-cannot  be  measured. 

o  The  prediction  employs  a  model  which  is  only  an  approximation  of 
actual  equipment  operational  conditions 
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f*  Bayesian  Approach  to  Reliability  Assessment  (References  3,  4,  and  5) 
(1 )  Introduction 


The  Bayesian  approach  to  reliability  assessment  is  an  approach  which 
can  be  applied  to  reliability  demonstration,  since  it  attempts  to 
incorporate  all  information  that  is  available  to  develop  ^stimat* 
of  the  reliability  of  the  equipment  under  analysis.  This  information 
includes  engineering  estimates  (predictions)  made  early  in  the  design 
c,_le  and  test  data  or  operational  data  collected  during  the  test  or 
operation  of  the  equipment.  The  predictions  are  sometimes  labeled 
“subjective  prior  estimates0  as  they  cannot  be  expressed  in  relative 
frequency  terms  (e.g.,  ratio  of  total  successes  to  total  trials).  The 
test  data  are  termed  "objective  data"  as  they  are  expressed  in  rela¬ 
tive  frequency  terms.  The  Bayesian  approach  to  reliability  assessment 
develops  a  rationale  which  permits  the  incorporation  oF  subjective 
Prior  estimates  with  objective  test  data  to  obtain  an  estimate  of  the 
reliability  of  the  equipment,  at  appropriate  statistical  confidence 
levels. 

The  Bayesian  approach  combines  the  early  reliability  assessment  with 
test  data  to  develop  early  estimates  of  equipment  reliability.  Early 
in  a  development  program,  wnen  test  data  are  not  available  on  a  parti¬ 
cular  design,  the  major  assessment  tool  is  the  reliability  prediction. 
As  the  program  continues,  sufficient  test  data  may  be  accumulated  such 
that  an  accurate  estimate  of  equipment  reliability  can  be  obtained 
based  solely  on  the  test  data.  This  point  usually  occurs  late  in  the 
development  cycle.  Therefore,  it  is  natural  to  develop  a  method  where 
uoth  the  prediction  (subjective  estimates)  and  objective  data  (test 
results)  can  be  combined  in  a  common  framework.  This  can  be  accom¬ 
plished  using  the  Bayesian  approach.  The  following  discussion  will 
develop  the  Bayesian  approach,  with  no  attempt  to  advocate  superiority 
of  the  approach  to  classical  techniques.  Also,  a  discussior  of  the 
soundness  of  combining  subjective  and  objective  information  will  not 
be  included.  Final  proof  of  the  applicability  of  the  Bayesian 
approach  will  be  obtained  through  field  experience. 

(2)  Bayes  Theorem 

Bayes  Theorem  is  an  analysis  technique  designed  to  bridge  the  gap 
between  prior  knowledge  and  statistically  conclusive  data.  The 
approach  permits  combination  of  prior  knowledge  with  test  data  making 
use  of  all  the  available  information  in  arriving  at  an  estimate  of 
equipment  reliability.  The  application  of  Bayes  to  reliability 
assessment  is  generalized  by  considering  the  following  quantities. 

y  =  The  total  estimate  of  the  reliability 

A  -  The  reliability  of  the  equipment  based  on  test  data 

B  =  All  prior  information  concerning  the  equipment  reliability 
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* 


(6) 


Using  these  terns,  Bayes  theorem  can  be  written: 

p  (r m  •  "l^8j^gj-VlfB) 


P  (y/AB)  ■  Probability  that  the  equipment  reliability  is  y, 

based  on  prior  estimates  and  test  data.  This  is  the 
desired  result. 

P  (y/B)  “  Probability  that  the  equipment  reliability  is  y, 

based  on  all  prior  information. 

P  (A/yB)  *  Probability  of  duplicating  the  test  data  when  the 
equipment  reliability  is  known,  along  with  prior 
information.  This  term  Is  the  estimate  of  equipment 
reliability  based  on  assjnption  of  an  underlying 
statistical  distribution,  (the  underlying  distribu¬ 
tion  which  is  assumed  is  typically  Poisson  or 
Binomial). 


P  (A/B)  *  /  P(A/.B)  .  P(y/B)  dY 

All  y  . 

«  Probability  of  duplicating  test  data  when  equipment 
reliability  is  unknown. 


Bayes  Theorem  is  then: 


p  -  4$)  M  d,  •  <7> 

All  Y 

The  next  step  in  the  formulation  process  is  to  select  suitable  prior 
distributions.  There  exist  no  explicit  rules  for  selecting  a  speci¬ 
fic  prior  distribution  [P  (t/B)].  The  following  criteria  can  be 
established. 

o  The  prior  distribution  must  adequately  reflect  what  is'  known  before 
test  data  is  available. 

o  The  prior  distribution  should  not  imply  any  assumptions  about 
unknown  Information  concerning  the  reliability  of  the  component. 

o  The  prior  distribution  should  "go  together"  with  the  existing  data 
but  allow  sufficient  variety  of  choice  so  that  any  reasonable 
prior  belief  can  be  represented  by  the  prior  distribution. 

o  Tne  resulting  expression  should  be  mathematically  tractable. 
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Although  the  above  criteria  are  general,  two  cases  of  specific  interest 
to  reliability  analysis  techniques  are  being  considered. 

o  If  the  sample  test  data  is  Poisson  distributed,  a  Gama  distribution 
is  a  convenient  prior.  A  Gama  distribution  will  be  obtained  as  a 
posterior  distribution. 

o  If  the  sample  data  is  binomially  distributed,  a  Beta  distribution 
is  most  conveniently  selected  as  a  prior.  A  Beta  Distribution  will 
result  as  a  posterior  distribution. 

Both  cf  these  situations  will  be  discussed  in  the  analysis. 

(3)  Bayesian  Approach  -  Gamma  Prior  Distribution 

The  Gama  distribution  is  used  as  a  prior  distribution  when  the  test 
data  is  considered  to  be  Poisson  distributed.  The  Gamna  prior  is: 


The  test  data  is  Poisson  distributed: 


P  (A/yB)  = 


P  (A/B) 


®  re 


•yT  ivt)*'1  e'n  (,t)f 


TPmrr 


dv 


(9) 

(10) 


After  suitable  mathematical  manipulation  P  (y/AB),  the  estimate  of  the 
equipment  reliability  considering  all  information  is: 


P  (y/AB) 


■(*+*.) 


<j>+f  r  -(YT+t)  v$+f*l 


[e 


t  =  Accumulated  test  time 
t  =  Pseudo  test  time 
f  =  Actual  failures 
$  =  Pseudo  failures 


1 A 


(ll) 
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The  pseudo  test  time  end  pseudo  failures  cm  be  considered  parameters 
of  the  prior  Gama  distribution. 

The  posterior  distribution  has  the  same  fora  as  the  prior  distribution. 
The  Initial  parameters,  *  and  t  are  replaced  by  *+f  and  r*-t.  The 
mean  and  standard  deviations  of  the  distributions  are  giver*  by: 


Prior  Posterior 


The  resulting  posterior  Gamma  distribution  includes  the  test  data 
(t  and  f)  end  the  Initial  estimate  (*  and  t).  The  Gamma  distribution 
can  be  used  to  obtain  confidence  limits  for  t,  based  on  prior  esti¬ 
mates  and  test  data. 


/esian  Approach  -  Beta  Prior  Distribution 


The  Beta  Distribution  is  used  as  a  prior  distribution  when  the  test 
data  is  considered  to  be  Binomial  distributed.  The  Beta  prior  is 
based  on  a  probability  P 


P  (P/B) 


(12) 


0<p  <1 

v,  o  >  0  where  p  is  pseudo  successes  and  v  is  pseudo  trials. 


The  test  data  is  based  on  s  successes  in  n  trials,  and  is  binomial!/ 
distributed. 


P  (A/pB) 


03) 


P  (A/B) 


dp 

04) 
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Finally 


?  {?/*£}  = 


f  1  ! 


-+S-1 


{1- 


P)* 


flK'i 
1  *  *  ) 


This  toe  is  a  Beta  distribution.  The  is ean  and  variance  are- 


Prior 


Posterior 


p 


P 


,+r.)c  (v+n-1 } 


The  .  -ta  distribution  can  be  used  to  deteroine  confidence  level  for  p, 
based  '■n  prior  esticates  and  test  data. 

(5)  Select  on  of  Pseudo  Test  Tiae  and  Failures 


The  previous  discussion  demonstrated  the  procedure  by  which  prior 
estimates  of  equipment  reliability  can  be  combined  with  test  data  to 
estate  the  reliability  of  the  equipment.  These  subjective  e^isates 
we^-e  called  pseudo  failures  and  test  time  (a  and  t).  It  is  necessary 
to  assign  values  to  each  of  these  quantities  and  care  must  be  exercised 
when  this  is  done.  If  s/r  =  .1  and  values  of  a  =  10  and  t  =  1000, 
were  used,  the  subsequent  test  data  may  not  be  sufficient  to  minimize 
the  effects  of  the  prior.  This  would  be  particularly  undesirable  if 
the  prior  estimates  and  the  test  data  lead  to  different  estimates  of 
the  true  failure  rate.  In  this  case,  it  is  desirable  that  the  prior 
estimate  is  eliminated  or  "washed  out"  by  the  actual  test  data. 

It  is  desired  then  to  establish  prior  estimates  that  are  "washed  out" 
by  the  test  data  as  more  data  is  accumulated,  particularly  if  the  test 
data  and  prior  estimates  lead  to  substantially  different  estimates  of 
the  failure  rate.  One  procedure  is  to  establish  an  uncertainty 
factor^  U: 


Y 


1 

T 


SI 


The  establishment  of  U  fixes  t,  *&lch  then  fixes  *,  since  2  prior 
estimate  of  r  is  assaaed. 


A  second  approach  would  be  to  select  ?  such  that  it  represented  a 
fraction  of  the  actual  test  tine  that  is  expected  tc  be  accural  a  ted. 
Then*  is  the  test  tixe  is  acaxpulated,  the  prior  estimate,  if  different 
fron  the  test  results,  will  "wash  out®.  If  the  prior  estimate  and 
test  data  are  caapatible  then  this  will  lead  wore  quickly  to  an 
estfaate  of  the  failure  rate.  If  a  Seta  distribution  was  assrjaed  as 
a  prior  distribution,  then  v  would  be  selected  as  a  fraction  of  the 
total  noaber  of  the  expected  test  trials.  As  test  data  is  accumulated 
the  estisate  is  either  reinforced  or  "wasted  out"  depending  cot  the 
accuracy  of  the  prior  estisate  and  the  test  results. 

(6)  Conclusions 


The  Bayesian  approach  is  designed  to  suoplenent  early  test  feta  to 
develop  esti rates  of  eqoipsent  reliability.  Later  in  the  progress, 
when  sufficient  test  data  are  available,  the  test  data  will  prtnride 
the  necessary  weans  by  which  the  equipsseat  reliability  can  be  estimated. 
The  initial  estimates,  if  in  error,  will  be  "washed  cut*  by  the  test 
data  and  the  reliability  estimates  will  be  based  upon  the  test  data 
alone.  To  be  assured  of  this  situation  the  estimates  of  pseudo  tiss> 

(t)  and  pseudo  failures  ($)  should  be  selected  such  that  -  is  selected 
to  be  only  a  fraction  of  the  expected  test  data.  Thus,  t  is  selected 
ir,  terns  of  assurance  that  it  will  not  be  a  factor  in  the  estimate  of 
equipment  reliability,  if  the  prior  estimate  proves  to  be  in  variance 
with  the  test  data. 

The  advantages  of  the  Bayesian  approach  are  as  follows.  In  the  early 
stages  of  attest  prograa,  cany  components  have  few  test  hours  and  no 
failures.  By  using  the  prior  estisate  of  the  component  reliability, 
it  is  possible  to  develop  reliability  estimates  on  each  component. 

Each  of  the  estimates,  if  no  failures  were  observed,  would  result  in 
reduction  of  the  equipment  failure  rate,  in  teres  of  both  the  predic¬ 
tion  and  the  test  data.  As  the  test  prograa  continues  and  sore  data 
is  acciraulated,  the  weight  of  the  test  data  will  be  the  determining 
factor  in  predicting  the  equipaent  reliability.  If  the  prior  estimate 
is  compatible  with  the  test  data,  the  effect  would  be  to  core  quickly 
obtain  failure  rate  estimates,  as  each  type  of  infonaation  contributes 
to  the  total  failure  rate  estimate.  If  the  prior  estimate  is  in 
variance  with  the  test  data,  the  prior  estimate  would  “wash  out,"  by 
virtue  of  the  procedure  used  to  determine  the  prior  estimate.  Thus, 
early  estimates,  which  may  be  incorrect,  will  not  be  a  significant 
factor  in  determining  the  final  component  failure  rates. 
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The  degree  which  the  early  predictions  or  prior  estimates  are  “washed 
out*  is  a  function  of  the  accuracy  of  the  early  predictions  and  the 
degree  *iich  they  duplicate  the  test  data.  If  the  predictions  favor¬ 
ably  caapare  with  the  ultimate  test  results,  the  Bayesian  approach 
will  allow  taore  rapid  estimates  of  the  reliability  to  be  developed. 

If  the  predictions  and  the  ultimate  test  data  are  in  variance,  the 
prior  will  almost  “wash  out"  completely  and  the  final  reliability 
estimates  will  be  based  or,  the  test  data. 

g.  Hatrix  Comparison  of  Four  Deaonstration  Techniques 

This  section  presents  a  cosparisor,  in  matrix  fora,  (Table  XLI X)  of  the 

four  deaonstration  tecnniques  discussed  in  the  previous  section.  The  four 

techniques  are  qualitatively  rated  in  accordance  with: 

o  Cost  -  Cgs t  of  running  the  test, 

o  Tiaeliness  -  tfhen  a  program  test  is  performed  and  whether 

increase  in  schedule  is  required,  and 

o  Assurance  Sained  -  Degree  of  assurance  that  results  of  test  accurately 

depict  equipment  reliability  under  service 
conditions. 


These  parameters  are  considered  the  most  significant,  from  a  comparison 
standpoint.  Since  tne  Bayesiar  approach  involves  all  techniques,  it  is 
not  included  in  the  matrices. 

h.  Proposed  Designs trat ion  Approach 

Based  upon  the  demonstration  technique  matrix  and  the  preceding  discussion, 
the  following  reliability  demonstration  approach  is  suggested: 

o  Employ  reliability  predictions  to  form  an  equipment  reliability  base¬ 
line.  Continually  update  the  predictions  as  new  data  becomes  avail¬ 
able  from  the  predesign  phase  through  the  period  of  field  operation 
and  maintenance. 

o  Develop  a  reliability  verification  test  program.  Define  assemblies 
which  require  reliability  verification  testing.  Implement  this  testing 
tc  minimize  and  eliminate  failure  mechanisms.  In  the  cases  where  com¬ 
ponents  exhibit  known  mechanisms  which  limit  the  life  of  the  component, 
test  the  component  to  better  understand  the  particular  mechanisms 
i nvol ved . 

o  Perform  a  formal  in-service  reliability  demonstration  to  determine 
reliability  under  use  conditions,  including  the  effects  of  maintenance 
policies  and  logistics  support.  Include  formal  accept/reject  criteria 
and  associated  penalties  a»id  rewards. 
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The  suggested  reliability  demonstration  approach  incorporates  features  of 
the  demonstration  by  prediction,  verification  testing,  and  in-seryice 
testing  techniques,  but  does  not  include  the  formal  in-plant  demonstration 
technique.  The  formal  in-plant  demonstrati  on  technique  is  excluded 
because  it  will  generally  be  the  most  costly  approach,  require  a  longer 
schedule,  and  the  results  will  have  dubious  correlation  with  the  actual 
in-service  measured  reliability. 

In  the  suggested  approach,  the  reliability  prediction  is  used  as  a  base¬ 
line  to  establish  the  initial  estimate  of  the  equipment  reliability.  This 
estimate  is  performed  at  the  system  level  ia  defines  the  expected  level 
of  reliability.  This  estimate  is  sot  -'ccepted  as  proof  of  reliability, 
but  only  serves  as  a  starting  point  in  the  demonstration  process.  As  the 
program  progresses,  the  prediction  will  be  continually  updated,  incorpora¬ 
ting  design  and  testing  information  that  is  generated,  thereby  increasing 
confidence  in  the  prediction. 

Ccncurrent  with  the  equipment  prediction,  a  list  of  components  and  assem¬ 
blies  is  identified  containing  candidates  for  verification  testing.  For 
each  component,  the  quantity  to  be  tested,  test  conditions  and  test  time 
is  determined.  These  components  will  be  selected  based  on  at  least  one 
of  the  following  criteria: 

o  High  design  risk 

State  of  the  art  design 

Known  history  of  high  failure  rate  on  similar  designs 
o  Known  failure  mechanisms 

o  Known  wearout  mechanisms  which  are  imperfectly  understood 

o  Designs  which,  based  on  a  failure  modes  and  effects  analysis, 
possess  critical  failure  mechanisms 

Tests  will  be  conducted  on  each  of  the  selected  components  and  assemblies 
aimed  at  determining  and  eliminating  the  failure  mechanisms.  In  the 
situations  where  the  components  exhibit  known  wearout  mechanisms,  these 
components  will  be  tested  to  obtain  a  better  understanding  of  these 
mechanisms.  The  tests  will  be  conducted  under  varying  degrees  of  st^sses, 
to  more  accurately  simulate  the  actual  operating  environment.  If  a  suffi¬ 
cient  sample  and  test  time  is  required  for  the  tests,  statistically 
meaningful  data  will  be  generated.  This  data  will  be  reduced  and  updated 
estimates  of  component  and  assembly  failure  rates  will  be  obtained.  These 
will  be  incorporated  in  the  updated  predictions. 

After  the  equipment  is  installed  and  debugged  in  the  field,  the  formal 
in-service  demonstration  test  will  commence.  This  test  would  be  performed 
concurrently  with  performance  and  evaluation  tests.  The  test  would  be 
performed  under  the  cognizance  of  contractor  and  customer  personnel  and 


95 


would  be  designed  to  formally  establish  the  achieved  reliability  of  the 
equipment.  Including  the  effects  of  envi forwent,  operation,  maintenance s 
and  logistic  policies.  The  results  of  the  test  will  provide  a  formal 
proof  of  equipment  reliability  under  use  conditions. 


SECTION  VIII 


SPECIAL  STUDIES 


INTRODUCTION 

The  prediction-by- function  technique  established  that  relationships  exist 
between  observed  reliability  and  selected  functional  parameters  for  a 
number  of  types  of  electronic  equipment.  In  addition,  studies  (1)  have 
shown  that  prediction  accuracy  varies  between  functional  types  of  equip¬ 
ment.  Therefore,  classification  and  grouping  of  equipment,  in  accordance 
with  its  function,  was  a  necessary  consideration  in  the  study  of  relia¬ 
bility  prediction  techniques. 

In  addition,  it  was  observed  that,  when  equipments  are  divided  into 
functional  groupings,  they  can  exhibit  a  wide  range  of  possible  design 
approaches  to  a  given  function.  Also,  within  any  one  design  approach,  a 
wide  variety  of  part  technologies  can  be  represented  (e.g.,  CRT  tubes, 
integrated  circuits,  solid  state  devices,  and  conventional  electronic 
parts).  Therefore,  it  was  also  necessary  to  establish  design  approach 
categories  and  part  technology  groupings  for  consideration. 

Equipments  were  divided  into  broad  functional  categories  (e.g.,  receiver, 
transmitter,  display,  and  computer  types).  These  categories  proved  useful 
for  such  purposes  as  analyzing  the  distribution  of  equipment  failure  rates 
over  long  periods  of  time,  and  for  comparing  observed  and  predicted 
failure  rates  for  functional  types.  (See  Special  Study  No.  1.)  In 
addition  to  broad  classifications,  categorization  by  sub-functions  (e.g., 
control,  power  supply,  synchronizer)  was  also  found  to  be  a  necessary  and 
useful  method  of  analysis  (see  Special  Study  No.  2). 

Several  methods  for  categorization  of  design  approaches  wer?  investigated. 
The  initial  method  attempted  to  align  design  approach  categories  with 
eauipment  functions,  since  some  functions  (e.g.,  digital  computei's  and 
pulse-doppler  radar  transmitters)  imply  established  design  characteris¬ 
tics.  However,  because  so  many  functions  can  be  accomplished  through  the 
use  of  a  wide  variety  of  designs  and  basic  hardware,  this  method  was  not 
used.  Furthermore,  many  functions  involve  a  hybrid  structure  in  terms  of 
the  various  part  technologies  which  are  used  in  the  design. 

Therefore,  a  second  method  considered  for  the  categorization  of  design 
approaches  was  based  upon  the  predominant  part  technology  used  for  the 
design,  (e.g.,  solid  state,  vacuum  tube,  integrated  circuit,  electro¬ 
mechanical,  magnetic,  etc.).  However,  correlation  analyses  failed  to 
establish  the  general  utility  of  these  categories.  Nevertheless,  it  was 
indicated  that  they  are  useful  as  sub-categories. 
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Evaluation  of  prediction  accuracy,  measured  In  terms  of  the  ratio  of  the 
observed  to  predicted  HTBF  values  (using  stress  analysis  prediction  tech¬ 
niques}  showed  two  distinct  groupings  of  equipment.  One  group  exhibited 
a  predominant  niaber  of  high  ratios,  while  the  other  showed  a  predominant 
master  of  low  ratios.  The  design  approach  categories  which  characterized 
these  groups  represented  “digital  and  analog"  equipments.  Within  the 
analog  group,  the  power  equipments  constituted  2  distinct  subset,  i.e., 
five  out  of  the  six  high-ratio  equipments  were  power  types.  However, 
within  the  digital  and  analog  groups,  there  were  no  other  distinct  sub¬ 
sets.  Further  analysis  of  prediction  inaccuracy  factors  failed  to 
indicate  any  further  relationships  traceable  to  design  approach.  There¬ 
fore,  the  categories  of  digital  and  analog  were  established  as  the  only 
design  approach  categories. 

The  Special  Studies  also  were  concerned  with  the  evaluation  of  many  other 
factors  pertinent  to  the  accuracy  of  the  reliability  prediction  process. 
These  included,  but  were  not  limited  to,  the  following: 

0  The  i^act  of  major  prediction  method  variables  such  as  stress, 
temperature,  and  part  grade  (see  Special  Study  No.  3); 

0  The  impact  of  program- related  factors  and  the  rating  of  these  factors 
(see  Special  Study  No.  4); 

0  The  effects  of  time  and  field  operations  upon  equipment  reliability 
(see  Special  Study  No.  5). 

2.  SPECIAL  STUDY  NO.  1:  "COtfARlSOH  OF  OBSERVED  AND  PREDICTED  FAILURE  RATES" 
a.  Object! ves 

The  objectives  of  this  Special  Study  are: 

0  Determine  if  thj  ratio  of  predicted  to  observed  failure  rate  (x  /A  ) 
could  be  categorized  for  particular  types  of  equipment  (i.e.,  p 
transmitter,  receiver,  computer  and  display  type  equipment).  That  is, 
does  the  ratio  A  /A  for  a  particular  type  equipment  possess  any  speci¬ 
fic,  di s ti ngui sh?ng°characteri s  ti  c  ? 

0  Define  a  possible  statistical  distribution  of  *pA0» 

0  Determine  a  method  that  could  be  employed,  using  the  data,  to  improve 
the  accuracy  of  predictions.  The  technique  used  in  this  Special  Study 
was  to  determine  a  division  factor  for  \  ,  such  that  the  ratio  of  A  /A 
would  be  either  within  plus  or  minus  10Cr%  of  >p/AQ  £  1,  or  p 

0.5  _<  Ap/A Q  _<  2.0 
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The  division  factor  (or  factors)  which  resulted  in  the  maximum  manber  of 

equipment  failure  rate  ratios  lying  in  this  range  would  be  selected  es 

representing  factors,  by  which  the  predicted  failure  rate  could  be 

divided,  resulting  in  "more  accurate"  equipment  predictions. 

b.  Summary 

In  summary,  the  salient  results  of  this  Special  Study  are: 

o  The  failure  rate  ratios  for  transmitter  equipments  analyzed  in  this 
study  (ad/a  )  are  all  less  than  0.5.  A  possible  implication  is  that 
predict!*  onsror  transmitter  equipment  tend  to  be  optimistic.  Failure 
rate  ratios  for  the  remaining  equipment'  types  show  that,  generally, 
predictions  are  pessimistic. 

o  The  probability  plot  of  failure  rate  ratios  is  not  log  normally  distri¬ 
buted.  A  normal  distribution  more  closely  fits  the  data. 

o  If  the  predicted  failure  rates  were  divided  by  a  constant  factor  in 
the  range  of  3.0  to  4.5,  (excluding  the  transmitter  and  camera  display 
equipments),  a  more  accurate  prediction  would  have  resulted.  Accuracy 
is  defined  as  the  predicted  failure  rate  deviating  from  the  observed 
failure  rate  by  no  more  than  plus  or  minus  100  percent,  which  can  be 
expressed  in  the  following  form: 

0.5  _<  Ap/AQ  jc  2.0 

o  Specifically,  if  a  factor  of  4.0  were  used  as  a  divisor  of  the 
predicted  failure  rate,  81  percent  of  the  failure  rate  ratios  lie 
within  0.5  _<  Ap/A0  _<  2.0  (The  transmitter  and  camera  equipment 
are  excluded.  In  fact,  a  multiplying  factor  of  4.0  would  be  appro¬ 
priate  for  these  equipments.) 

c.  Details 

(1)  Comparison  of  Observed  and  Predicted  Failure  Rate  -  Data 
Presentation 

The  predicted  failure  rate  for  each  equipment  was  obtained  from  field 
reports  and  generally  represents  the  early  contractor's  predictions. 
The  prediction  techniques  used  by  the  contractors  include  prediction 
by  similarity  and  prediction  by  parts  count.  The  predictions  employed 
in  the  study  varied  and  did  not  all  employ  the  same  techniques  (i.e.; 
not  all  predictions  used  the  stress  analysis  methods  described  in 
MIL-HDBK-217A) .  The  observed  failure  rate  uas  obtained  from  field 
data  taken  after  the  equipment  had  been  in  operation  in  the  field 
approximately  one  year. 
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Tables  L  through  LII1  present  a  comparison  of  contractors*  predicted 
failure  rates  to  observed  failure  rates.  The  comparison  is  made  at 
two  points  in  time:  (1)  at  the  end  of  a  12  month  period;  ar.d,  (2) 
at  the  end  of  27  months  (except  where  otherwise  indicated).  All 
comparisons  are  made  on  the  basis  of  fa^ures  per  1000  hours  per 
equipment  type.  Failure  rates  are  average  failure  rates  for  an  indi¬ 
vidual  unit  per  equipment  type.  That  is,  for  T-l  the  failure  rate 
listed  is  tha  average  failure  rate  of  the  six  transmitter  units  of 
that  equipment  type. 


TABLE  L 

TRANSMITTER  TYPE  EQUIPfCNT  -  FAILURE  RATE  COMPARISON 


Equipment 

Predicted  Failure  Rate 

( X/1000  hours)  >-p 

Observed  Failure 
Rate  at  12  Months 
(x/1000  hours) 

Observe^  Failure 
Rate  at  27  Months 
(x/1000  hours)  '■‘o 

Ratio 

Vxo 

T-l 

9.4 

28.2 

20.5 

0.46 

T-2 

5.4 

24.0 

16.2 

0.33 

T-3 

9.4 

16.2 

19.4 

0.48 

T-4 

5.4 

19.2 

_ 

15.5 

0.35 

Table  L  shows  that  all  four  of  the  observed  failure  rates  exceed  tne 
precicted  failure  rate  by  substantial  margins.  These  four  equipments 
are  high  power  transmitters  and  the  primary  contributors  to  the  trans¬ 
mitter  failure  rate  are  the  power  tubes  used  in  the  transmitter. 
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TABLE  LI 


RECEIVER  TYPE  EQUIPMENT  -  FAILURE  RATE  COMPARISON 


1 — 
Equipment 

— 

Predicted  Failure  Rate 

(x/1000  hours)  xp 

Observed  Failure 
Rate  at  12  Months 
{x/1000  hours) 

Observed  Failure 
Rate  at  27  Months 
(x/1000  hours)  x0 

Rati  o 

x'xn 

P  o 

R-l 

7.6 

1.4  ' 

1.1 

6.9 

R-2 

3.2 

0.8 

0.8 

4.0 

R-3 

0.6 

0.15 

0.15 

4.0 

R-4 

7.6 

1.5 

1.4 

5.4 

R-5 

3.2 

1.4 

1.5 

2.1 

R-6 

0.6 

0.11 

0.12 

5.0 

R-7 

1.9 

0.4 

0.4 

4.7 

R-8 

1.9 

0.6 

0.7 

2.7 

The  comparison  between  the  observed  and  predicted  failure  rate  shows 
that  the  predicted  failure  rate  exceeds  the  observed  failure  rate  in 
all  cases.  The  ratios  of  predicted  failure  rate  to  observed  failure 
rate  range  between  2.1  and  6.9,  with  an  average  of  4.35.  Table  LI 
shows  these  results. 


TABLE  LI  I 


COMPUTER  TYPE  EQUIPMENT  -  FAILURE  RATE  COW>ARISON 


Equipment 

Predicted  Failure  Rate 

(x/1000  hours)  *p 

- — r 

Observed  Failure 
Rate  at  12  Months 
(x/1000  hours) 

Observed  Failure 
Rate  at  27  Months 
(x/1000  hours)  *o 

Ratio 

y\> 

C-l 

Not  Available 

0.35 

0.36 

— 

C-2 

18.2 

7.5 

7.6 

2.40 

C-3 

Not  Available 

5.6 

4.3 

— 

C-4 

Not  Available 

0.5 

0.34 

— 

C-5 

1.8 

0.5 

0.4 

4.5 

C-6 

Not  Available 

0.5 

0.4 

— 

C-7 

18.2 

5.2 

5.4 

3.37 

C-8 

Not  Available 

1.5 

0.75 

— 

C-9 

Not  Available 

0.12 

0.10 

— 

C-10 

1.8 

1.8 

1.1 

1.64 

c-n 

Not  Available 

1.4 

★ 

1.5 

— 

C-12 

Not  Available 

2.3 

* 

3.2 

-- 

C-13 

5.0 

1.1 

* 

0.9 

5.5 

C-I4 

2.5 

3.1 

* 

2.5 

1.0 

C-15 

Not  Available 

Elec. 

0.6 

0.6  * 

Mech 

0.6 

0.6 

C-16A 

91 

2C.5 

21.5 

4.2 

*  18  nsonths 

A  C-16  *  C-11  +  C-12  +  14  (C-15)  ;  whore  there  are  14  unite  of 
equipment  C-15. 
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The  predicted  failure  rate  was  only  available  for  six  of  the  fifteen 
computer  type  equipments,  which  represent  approximately  20%  of  the 
computer  equipment  hours  accumulated,  but  approximately  45%  of  the 
total  failures  accumulated.  In  all  the  cases,  the  predicted  failure 
rate  was  greater  than  the  observed,  by  a  factor  that  ranges  from  2 
to  approximately  5. 

A  prediction  was  performed  by  the  contractor  for  the  computer  complex 
which  includes  C-ll ,  C-12  and  C-15.  This  is  shown  as  C-16.  This 
predicted  failure  rate  is  considerably  higher  than  the  observed,  after 
combining  the  appropriate  data.  Table  LI I , "Computer  Type  Equipment 
Failure  Rate  Comparison,1,  presents  the  data  on  all  the  computer 
equipment. 


TABLE  LI I I 

DISPLAY  TYPE  EQUIPMENT  -  FAILURE  RATE  COMPARISON 


Equipment 

Predicted  Failure  Rate 

(a/ 1000  hours)  Xp 

Observed  Failure 
Rate  at  12  Months 
(x/ 1000  hours) 

Observed  Failure 
Rate  at  27  Months 
(a/1000  hours)  xo 

Rati  o 

A  /A 
p'  0 

D-l 

1.75 

0.53 

0.34 

5.15 

0-2 

4.65 

1.3 

0.70 

6.65 

D-3 

Not  Available 

0.15 

0.10* 

— 

D-4 

1.75 

0.35 

0.50 

3.50 

D-5 

4.65 

2.0 

1.3 

3.57 

D-6 

Not  Available 

0.04 

0.03 

*** 

D-7 

5.70 

3.1 

3.0 

1.90 

D-8 

1.15 

15.6 

— 

0.07® 

D-9 

7.40 

4.2 

— 

1.76® 

*  24  months  @  Calculated  using  failure  rate 

**  15  months  at  end  of  12  months. 

***  18  months 


The  observed  failure  rav.e,  in  every  case  but  one,  was  less  than  the 
predicted  failure  rate.  This  case  was  D-8,  the  camera  equipment 
which  had  components  with  known  reliability  limitations.  This  is 
shown  in  the  above  table. 
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(2)  Failure  Rate  Comparison  -  Predicted  vs.  Observed  -  Results 

Table  LIV  presents  the  results  of  the  failure  rate  comparisons  by 
equipment  type  as  shown  in  the  four  preceding  tables.  It  was  possible 
to  obtain  predicted  failure  rates  on  26  of  the  36  equipments.  These 
predictions  were  performed  by  contractor  early  in  the  dcvelopc^nt 
pycle.  Therefore,  they  represent  the  initial  estimates  of  the  equip¬ 
ment  failure  rate.  The  prediction  methods  included  a  prediction  by 
similarity  method  and  a  parts  count  method.  The  observed  failure 
rate  Included  failures  due  to  all  causes  including  design  anomalies, 
wearout,  maintenance  policies  and  random  failures  in  time. 


TABLE  LIV 

COMPARISON  OF  PREDICTED  AND  OBSERVED  FAILURE  RATES 


» 

RATIO  OF  PREDICTED  FAILURE  RATE  TO  OBSERVED  FAILURE 

RATE  . 

Xo 

EQUIPMENT  ' 

0.05- 

0.51- 

lTOT^ 

2.01- 

5. 01- 

II  V 

O 

o 

TYPE 

0.5 

1.00 

2.00 

5.00 

10.00 

Transmitter 

4 

0 

0 

0 

0 

o  1 

Receiver 

0 

0 

0 

6 

2 

0 

Computer 

0 

1 

1 

4 

1 

0 

Display 

1 

„  A  . 

L  2 

.  A  „ 

2 

_  A  - 

TOTALS 

5 

-A. 

3 

_  12_ 

5 

_  A  _ 

PERCENTAGE  OF  TOTAL 
(26  EauiDments) 

19,2 

3.8 

11.6 

46.2 

IS. 2 

0 

The  table  presents  the  ratio  of  the  predicted  failure  rate  to  the 
observed  failure  rate  for  the  26  equipments  per  equipment  type.  The 
table  further  subdivides  the  ratio  into  six  groups,  ranging  from 
predictions  which  were  one-twentieth  of  the  observed  failure  rate  to 
predictions  which  were  ten  times  the  observed  i.ilure  rate.  Consider¬ 
ing  the  equipment  types  as  one  group,  15.4%  or  tne  ratios  fall  between 
plus  and  minus  1 00%  of  the  observed  failure  rate.  The  remaining 
84.62  are  either  less  than  0.5  (>.  />.  <0.5)  or  are  greater  than  2.0 
( x  / a  >  2.0).  None  of  the  ratios  exceeded  10.0.  Figure  2  shows 
the  cumulative  plot  of  the  ratio  on  log  probability  paper.  If  the 
distribution  of  X  predicted/x  observed  were  log  normal,  this  plot 
could  be  fitted  by  a  straight  line.  However,  as  can  be  seen,  a 
straight  line  would  be  a  poor  fit. 

The  chi-square  goodness  of  fit  test  was  applied  to  the  logarithms  of 
the  daca  to  determine  if  the  hypothesis  that  the  data  could  be  fitted 
with  a  log  normal  distribution  could  be  accepted.  Chi-square  calculated 
to  be  8.56  with  one  degree  of  freedom.  The  value  of  chi-square  at  one 
degree  of  freedom  at  the  992  confidence  level  is  6.635.  The  calculated 
value  of  chi-square  exceeds  this  valie  and  it  can  be  concluded  that  the 
data  points  do  not  fit  a  log  normal  distribution.  Hence  the  hypothesis 
that  the  log  normal  distribution  is  an  adequate  fit  for  these  points  is 
rejected. 

The  chi-square  test  was  then  applied  to  the  actual  ratios  to  test  the 
hypothesis  that  the  ratios  could  be  fitted  by  a  normal  distribution. 

All  data  were  included  except  the  camera  equipment.  The  value  of  chi- 
square  calculated  from  the  data,  with  one  degree  of  freedom  is  1.25. 

This  does  not  exceed  the  value  of  chi-square  in  the  table,  3.84,  at  952 
confidence  and  one  degree  of  freedom.  Thus,  the  normal  distribution 
cannot  be  rejected  as  a  fit  for  the  data.  Figure  3  plots  the  data  on 
normal  probability  paper. 

Table  LI V  shows  that  the  observed  failure  rates  for  all  four  high 
power  transmitters  were  greater  than  the  predicted  failure  rates  by 
more  than  1002.  Only  one  other  equipment  displayed  this  behavior, 
a  camera  projection  equipment  in  the  display  group.  For  the  receiver 
type  equipment,  all  the  equipment  had  predicted  failure  rates  that 
exceeded  the  observed  failure  rates  by  a  factor  of  at  least  two,  and 
33.32  of  the  equipment  had  predicted  failure  rates  that  exceeded  the 
observed  failure  rates  by  a  factor  of  at  least  five.  For  the 
computer  equipment,  71.42  of  this  group  had  predicted  failure  rates 
which  exceeded  the  observed  failure  rates  by  a  factor  of  at  least 
two  times  the  observed  failure  rate.  For  the  display  equipment, 
discounting  the  camera  projector,  672  of  the  equipment  had  predicted 
failure  rates  which  exceeded  the  observed  failure  rates  by  a  factor 
of  two  (Ap/xo  >_  2.0). 
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Figure  2.  Log  Normal  Probability  Plot  of  Ratio  X  /x 
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Table  UV  shows  that  46.2  per  cent  of  the  equipment  failure  rate  ratios 
(ipA0)  lie  In  the  range  of  2.01  to  5.00.  This  suggests  that  a  factor 
could  be  applied  to  the  predicted  failure  rate  such  that  the  predic¬ 
tion  would  better  approximate  the  observed  results.  To  determine  the 
acceptability  of  this  factor, an  acceptability  criteria  is  required. 

Ip  this  study,  the  acceptability  criteria  is  established  as  follows: 

If  the  ratio  xD/x0  is  between  .5  and  2.0,  the  prediction  is  defined 
as  an  acceptable  estimate  of  the  true  failure  rate.  Mathematically: 

.5  <  An/x  <  2.0 
—  p  o  — 

The  technique  employed  In  the  study  is  to  divide  the  predicted  fail¬ 
ure  rate  by  a  constant  and  recompute  Xn/x0.  As  a  larger  number  of 
these  ratios  fall  within  the  acceptable  range  defined  above,  the  pre¬ 
diction  is  judged  to  be  more  acceptable.  Figure  4  shows  the  relation¬ 
ship  between  the  division  factors,  ranging  from  2.0  to  7.0,  and  the 
number  of  equipment  failure  rate  ratios  which  fall  within  the  defined 
acceptable  range. 

Discounting  the  four  transmitter  and  the  camera  display  equipments, 
(since  division  of  XD  by  a  constant  would  result  in  a  more  erroneous 
ratio)  85.6  percent  bf  the  equipment  failure  rate  ratios  lie  within 
0.5  to  2.0  when  division  factors  of  3.0  or  3.5  are  applied.  If  a  divi¬ 
sion  factor  of  4.0  is  used,  then  81.0  percent  of  the  equipment  failure 
rate  ratios  lie  within  the  range  defined  above.  If  a  division  factor 
of  4.5  is  used,  76.4  percent  of  the  equipment  failure  rate  ratios  lie 
within  this  range.  Thus,  the  prediction  accuracy  for  this  equipment 
could  be  improved  if  a  division  factor  between  3.0  and  4.5  were  used 
as  a  divisor  for  the  predicted  failure  rate  originally  developed. 

Table  LV  is  developed  using  a  dividing  factor  of  4.0.  All  the 
failure  rate  ratios  now  lie  between  0.02  and  2.00.  Discounting  the 
transmitter  and  camera  equipments,  all  failure  rate  ratios  lie  between 
0.25  and  2.0. 
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Figure  4.  Number  of  Equipment  Groups  Falling  in  Failure  Rate  Ratio 
Range  of  0.5  <  \  A  <  2.0 

—  p  o  — 
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TABLE  LV  RATIO  OF  PREDICTED  FAILURE  RATE  DIVIDED  BY  FOUR,  TO  OBSERVED  FAILURE  RATE 


3.  SPECIAL  STUDY  HO.  2: 
a.  Objectives 


"ANALYSIS  OF  PREDICTION  INACCURACY  FACTORS" 


The  objectives  of  this  study  were  the  identification  and  analysis  of 
prediction  inaccuracy  factors  inherent  to  the  following  categories: 

o  Design  Approach 

o  Equipment  Function 

o  Program  -  Related  Factors 

o  Parts  -  Mix 

o  Prediction  Method 

b.  Summary 

The  analysis  uf  the  prediction  inaccuracy  factors  resulted  in  the 
following  observations: 

o  The  design  approach  categories  (i.e.»  digital  and  analog)  exhibit 
different  prediction  accuracy  ratios  and  part  of  this  difference  can 
be  ascribed  to  design  characteristics. 

o  The  equipment  function  categories  (i.e.,  power, test  and  display,  con¬ 
trol,  and  computers)  have  different  prediction  accuracy  ratios, and 
part  of  these  differences  can  be  attributed  to  function  and  active 
element  content, 

o  The  reliability  program  factors  cannot  be  isolated  as  contributions  to 
individual  equipment  prediction  accuracy. 

o  The  prediction  accuracy  ratio  of  an  equipment  is  influenced  by  the 
types  of  parts  predominant  in  the  design. 

o  An  error  in  selecting  either  stress,  temperature,  or  part  quality 
values  in  the  prediction  process  could  contribute  significantly  to 
prediction  inaccuracies. 

o  The  semiconductor  application  factor  UJ  is  a  significant  contributor 
to  prediction  inaccuracy. 
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c.  Details 


(1)  Introduction 

The  analysis  of  prediction  inaccuracy  factors  was  performed  for  each 
of  the  following  categories: 

o  Design  Approach 

o  Equipment  Function 

o  Program  -  Related  Factors 

o  Parts-Mix  Factors 

o  Prediction  Method  Factors 

For  purpose*  of  this  study,  prediction  accuracy  is  defined  as  the 
ratio  of  the  observed  MTBF  to  the  RADC  Volume  II  upper  grade  part 
predicted  MTBF.  Obviously,  the  most  desirable  prediction  accuracy 
value  is  1.0.  The  analysis  performed  on  the  above  factors  and  the 
results  of  the  analyses  are  contained  in  the  following  subsections. 

(2)  Design  Approach  Categories 

The  design  approach  analysis  considered  variations  in  prediction 
accuracies  which  exist  between  the  two  design  approach  categories 
discussed  in  VIII  1.  above,  l.e.,  digital  and  analog  equipments.  The 
distributions  of  prediction  accuracy  ratios  for  the  14  digital  and  20 
analog  equipments,  which  w*re  utilized,  are  shown  in  Figure  5, 
"P-ediction  Accuncy  Ratio  Distributions  for  Analog  and  Digital 
Equipment." 

For  the  digital  equipment  group,  the  mean  of  all  the  prediction 
accuracy  ratios  is  4.1,  with  a  standard  deviation  of  3.2.  The  range 
of  the  digital  prediction  accuracy  ratios  is  from  0.7  to  11.4  (i.e., 
about  16.5  to  1).  There  are  only  two  digital  ratios  below  the  desired 
1.0  value  (i.e.,  0.7  and  0.9).  All  other  digital  ratios  exceed  the 
desired  1.0  value. 

The  analog  equipment  group  prediction  accuracy  ratios  have  a  mean  of 
3.2,  a  standard  deviation  of  6.5,  and  range  from  0.06  to  26.5  (i.e., 
about  442  to  1).  Fourteen  analog  equipments  ratios  were  less  than  the 
desired  1.0. 

From  the  above  It  is  concluded  that  digital  equipments  predominantly 
have  low  predicted  MTBF's  when  compared  with  the  observed  values,  and 
analog  equipments  predominantly  exhibit  high  predicted  MTBF's  when 
compared  with  the  observed  values.  Some  of  the  design  characteristics 
which  account  for  these  anomalies  are: 

o  Digital  equipments  operate  on  a  low  power  requirement  while  analog 
equipment  power  requirements  extend  over  a  large  range, 
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Prediction  Accuracy  Ratio 


Figure  5.  Prediction  Accuracy  fatio 
Distributions  for  Analog 
and  Digital  Equipment 


o  Digital  equipments  are  bi-stable,  i.e.,  operate  at  one  of  two 
levels  cr  states. 

o  Analog  equipments  operate  continuously  over  an  infinite  ranae 
and  are  thus  susceptible  to  additional  failure  modes,  e.g., 
drift  sensitivity,  etc. 

(3)  Equipment  Function 

In  assessing  the  relationship  of  functional  characteristics  to  pre¬ 
diction  accuracy  the  equipments  were  divided  into  four  groups: 
power,  control,  computer,  and  test  and  display.  There  were  six 
power  equipments,  seventeen  control  equipments,  five  computer  equip¬ 
ments  and  six  test  and  display  equipments. 

The  six  power  equipments  have  prediction  accuracy  ratios  ranging 
from  1.3  to  26.5.  That  is,  all  the  MTBF  predictions  are  low  when 
compared  to  the  observed  values.  The  six  power  equipments  are, 
basically,  all  power  supplies.  However,  two  equipments  have  test 
functions  as  part  of  their  operation.  These  two  equipments  also 
have  the  best  prediction  accuracy  values  {i.e.,  1.3  and  1.5). 

The  mean  of  the  prediction  accuracy  ratios  for  all  the  equipments  is 
3.5,  whereas  the  mean  for  the  six  power  equipments  is  significantly 
higher,  9.5  (i.e.,  almost  1000  percent  above  the  desired\alue). 
However,  there  is  no  evidence  that  the  high  prediction  accuracy  ratios 
can  be  ascribed  to  other  factors  such  as:  parts  count,  active  ele¬ 
ments,  or  derating.  Therefore,  in  these  cases  the  prediction 
inaccuracy  was  ascribed  to  unidentified  factors  associated  with  the 
configuration  of  power  function  equipment. 

The  seventeen  control  equipments  have  prediction  accuracy  ratios 
ranging  from  C.06  to  11.4  with  a  mean  of  2.8.  Although  this  mean  is 
closer  to  the  desired  prediction  accuracy  of  1.0  than  the  mean  for  all 
equipments  (3.5),  eleven  of  the  control  equipments  have  prediction 
accuracy  values  of  less  than  1,0.  In  fact,  only  four  of  the  seventeen 
equipments  are  close  (0.7,  0.7,  0.8,  and  0.9)  to  the  desired  value. 
Four  of  the  equipments  exhibited  extreme  Inaccuracies  of  1000  percent 
or  better  (i.e.,  0,07,  0.09,  0.09,  and  11.4).  The  eleven  control 
equipments  which  have  prediction  accuracy  values  less  than  the  desired 
value  of  1.0,  and  the  six  which  had  ratios  above  1.0,  were  investi¬ 
gated  separately  and  collectively  to  determine  if  such  factors  as  part 
count,  active  element  count,  and  solid  state  part  count  contributed  to 
their  inaccuracy.  No  significant  correlation  was  found  between  any  of 
the  above  factors  and  the  prediction  accuracy  ratios.  However,  it  was 
noted  that  all  3  equipments  using  integrated  circuits  had  high 
ratios  whereas  all  5  equipments  using  vacuum  tubes  had  low  ratios. 
Thus,  approximately  >alf  r  the  control  function  equipment  inaccuracy 
was  determined  to  be  part-related.  The  remainder  was  attributed  to 
other  unidentified  factors  associated  with  the  control  equipments. 
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The  five  computer  type  equipments  have  prediction  accuracy  values  of 
1.7,  1.7,  1.8,  2.1,  and  3.8.  The  equipment  with  the  highest  predic¬ 
tion  accuracy  ratio  (3.8)  includes  a  parts  complement  of  33  percent 
integrated  circuits  and  27  percent  other  active  element  devices. 

Also,  the  equipment  with  the  prediction  accuracy  value  of  2.1  includes 
a  parts  complement  of  over  50  percent  active  element  devices.  The 
other  three  equipments  contain  less  than  50  percent  active  element 
devices.  Hence,  there  is  an  evident  relationship  between  high  pre¬ 
diction  accuracy  ratios  and  the  active  element  content  of  computer 
type  equipments. 

The  six  test  and  display  equipments  have  prediction  accuracy  ratios  of 
0.06,  0.09,  0.09,  0.9,  0.9,  and  2.2.  The  three  equipments  with  the 
extremely  low  prediction  accuracy  ratios  have  a  lower  acti  /e  element 
mix  than  the  other  three  equipments.  The  active  elements  in  the  three 
lowest  valued  equipments  Include  vacuum  tube  components.  Thus,  low 
active  element  ratios  were  significant  contributors  to  prediction  in¬ 
accuracies  for  the  test  and  display  functional  types. 

(4)  Prog ram- Re la ted  Factors 

The  investigation  of  inaccuracies  in  the  prediction  techniques  which 
are  program-related  is  contained  in  the  rating  studies  in  Special 
Study  No.  4,  Section  VIII.  The  rating  studies  indicated  that  the  pro¬ 
grams  with  rating  values  higher  than  3.55  had  system  prediction 
accuracy  ratios  greater  than  1.0.  Hence,  if  all  the  program  factors 
were  maximized,  the  prediction  techniques  would  result  in  a  higher 
prediction  accuracy  ratio.  However,  it  WuS  also  shown  in  the  rating 
studies  that  a  prediction  modifier  could  be  established  which  would 
impi'-ove  the  accuracy  of  a  system  p"ediction  but  would  not  improve  the 
accuracy  for  the  indiyid-.J  equipme-t.  Therefore,  the  effects  of 
program-related  factors  are  not  discernible  in  the  analysis  of 
individual  prediction  accuracy  ratios, 

(5)  Parts-Mix  Factors 

The  inaccuracies  of  the  prediction  technique  ./ere analyzed  to  determine 
if  relationships  existed  with  respect  to: 

o  Part  Count 

o  Active  Element  Count 

o  Part  Type  Factors 

The  active  element  count  expressed  as  the  ratio  of  total  transistors, 
diou.-s,  and  vacuum  tubes  to  total  part  count,  was  correlated  with  pre¬ 
diction  accuracy  ratios.  The  correlation  analysis  was  performed  for 
three  equipment  categories  ( i . e . ,  all  34  equipments,  digital  equipment 
and  analog  equipment).  The  results  of  this  analysis  is  shown  in 
Table  LVI,  "Correlation  of  Active  Elements  versus  the  Prediction 
Accuracy  Values." 
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TABLE.  LVI  CORRELATION  OF  ACTIVE  ELEMENTS 
VS  PREDICTION  ACCURACY  VALUES 


Category 

Correlation 
Coefficient  (R) 

Index  Of 

DetenTii nation  (R2) 

All  Equipment 

0.48 

0.232 

Digital 

-  _  _  - 

0.56 

0.314 

Analog 

0.43 

0.186 

The  table  shows  that  meaningful  correlation  does  not  exist  between  the 
active  element  components  and  the  prediction  accuracy  ratios  Hence, 
it  is  concluded  that  all  the  prediction  inaccuracies  are  not  assigna¬ 
ble  to  tne  active  element  components. 

Similarly,  correlation  analyses  were  conducted  for  the  same  three 
equipment  categories  between  the  equipment  part  count  and  the  predic¬ 
tion  accuracy  ratios.  The  results  indicated  that  no  relationships 
existed  (i.e.,  less  than  0.2  for  all  three  categories). 

The  part  type  analysis  attempted  to  relate  the  inaccuracies  of  the 
prediction  technique  to  the  types  of  part  predominant  in  the  design. 
The  following  types  of  parts  were  investigated: 

o  Tubes 

o  Capacitors 

o  Resistors 

o  Di odes 

o  Transistors 

o  Integrated  Circuits 

The  equipments  were  ordered  by  prediction  accuracy  ratios  (highest 
order  first),  and  by  content  of  the  individual  part  types.  The  order 
of  the  prediction  accuracy  ratios  was  compared  with  the  order  of  the 
part  type  content.  From  this  comparison  the  following  conclusions 
were  established: 

o  Tube  type  devices  contribute  significantly  to  a  lower-than- 
desired  prediction  accuracy  ratio. 
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Capacitors  do  not  contribute  significantly  to  any  variation 
of  the  prediction  accuracy  ratios. 

o  Diodes  and  resistors  contribute  to  lower-than-desired  prediction 
accuracy  ratios. 

o  Transistors  and  integrated  circuits  contribute  to  a  higher- 
than-desired  prediction  accuracy  ratio. 

(6)  Prediction  Method  Factors 

Factors  in  the  RADC  stress  analysis  prediction  technique  were 
evaluated  to  determine  if  the  accuracy  of  predictions  for  indi¬ 
vidual  equipments  could  be  related  to: 

o  Part  quality  grade 

o  Stress  effects 

o  Temperature  effects 

o  Failure  rate  modifiers 

:  Special  Study  No,  3.  Section  VIII  evaluated  the  effects  of 
stress,  temperature,  and  part  quality  by  creating  prediction 
variability  envelopes  representing  a  range  of  influence  for  these 
factors  on  the  variability  of  prediction  techniques.  By  com¬ 
paring  the  data  with  these  envelopes,  it  was  determined  that  19 
of  the  34  equipments  had  observed  values  which  were  within  the 
range  of  their  variability  envelopes.  Fifteen  observed  values 
were  outside  the  range.  However,  if  new  variabilit>  envelopes 
are  prepared,  using  the  extreme  values  of  stress  and  temperature 
from  the  RADC  Notebook,  Volume  II,  the  observed  values  for  29 
of  the  34  equipments  will  fall  within  the  range  of  the  prediction 
technique.  Therefore,  under  worst-case  and  best-case  conditions, 
it  would  be  possible  for  the  prediction  technique  to  encompass 
the  observed  values  for  all  but  five  equipments. 

However,  since  these  extremes  of  stress  and  temperature  are 
definitely  not  present  in  the  equipments,  only  the  original 
variability  envelopes  warrant  consideration.  Thus,  based  on  the 
sensitivity  study,  it  is  possible  that  the  prediction  inaccura¬ 
cies  associated  with  19  equipments  could  be  due  to  the  effects 
of  errors  in  estimating  stress,  temperature,  and  part  quality 
grade. 

However,  other  factors  in  the  prediction  technique  must  also  be 
considered.  The  effects  of  the  application  factor  (n^)  on  transistor 
and  diode  failure  rates  in  digital  and  analog  equipment  were 
evaluated  tc  determine  their  contribution  to  the  total  equipment 
failure  rates.  The  effect  of  this  factor  on  digital  equipment 
is  to  cause  a  decrease  in  the  total  equipment  failure  rate  by  an 
amount  to  be  determined  by  the  quantity  of  semiconductors  of  each 
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type  and  the  ratio  of  their  failu  e  contribution  to  the  total 
equipment  failure  rate.  Predictions  for  samples  representing 
a  range  of  analog  and  digital  equipments  were  evaluated  to 
determine  the  impact  of  this  factor  upon  th*  total  equipment 
failure  rate.  The  results  are  s* mrcarized  •>  the  following  Table 
LYII,  ''Impact  of  Semiconductor  Application  'actor  (it.)  on  Equip¬ 
ment  Predictions." 


TABLE  LVII 

IMPACT  OF  SEMICONDUCTOR  APPLICATION 
FACTOR  (nA)  ON  EQUIPMENT  PREDICTIONS 


Equipment 

Code 

Number 

Equipment 

Type 

Total 

Part 

Count 

(1000‘s) 

Per  Cent 
Semicon¬ 
ductors 

— 

Per  Cent 
Increase  (+) 
or 

Decrease  (-) 
in 

Failure  Rate 

3 

Analog 

.3 

30.0 

-39 

11 

Analog 

3.8 

28.9 

-29 

22 

Analog 

28.0 

30.8 

-75 

37 

MBEM 

35.0 

41.2 

+106 

1 

4.3 

46.0 

+109 

1G 

1 . .  M  ■  .1  ■ 

Hal 

2.1 

53.3 

+  92 

The  table  shows  that  the  failure  rates  of  analog  equipment  would 
decrease  by  from  29  to  75  percent  if  this  factor  was  applied, 
however,,  the  prediction  accuracy  ratios  indicate  the  predicted 
failure  rates  a^e  generally  too  low  without  this  factor.  The 
digital  equipment  failure  rates  would  approximately  double  if 
this  factor  was  not  applied.  Howeyer,  the  prediction  accuracy 
ratios  indicate  that  the  predicted  digital  equipment  failure 
rates  are  already  too  high  in  almost  all  cases  -  i.e.,  the 
aoplication  factors  are  also  too  high.  Therefore,  it  was  con¬ 
cluded  that  the  application  factor  was  a  significant  so*rce  of 
prediction  inaccuracy  for  digital  equipment.  For  analog  equip¬ 
ment,  an  appl ication  factor  which  increases  the  failure  rate 
would  offer  a  potential  for  improvement  in  prediction  accuracy. 
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4.  J>HLCi.V  STiJOY  NO.  3:  " SENSITIVITY  ffUDY" 

a.  Objectives 

This  strdy  was  performed  to  evaluate  the  relative  impact  of  major  elements 
of  the  RADC  Reliability  Notebook.,  Volume  II  stress  analysis  prediction 
technique  upon  the  variability  of  reliability  predictions  for  a  variety  of 
equipment  types.  Principal  objectives  were: 

o  Determine  the  amount  of  variation  in  predicted  failure  rates  contri¬ 
buted  by  the  prediction  elements  of  stress,  temperature,  and  part  grade. 

o  Determine  if  the  differences  between  observed  equipment  failure  rates 
and  predicted  failure  rates  fall  within  a  range  of  variation  in  pre¬ 
dicted  values  resulting  from  the  effects  of  stress,  temperature  and 
part  grade. 

b.  Sumnary 

The  study  showed  that  the  prediction  clement,  part  quality  grade,  provided 
the  most  significant  single  contribution  to  prediction  variation.  For 
both  digital  and  analog  equipments,  this  factor  could  account  for  approxi¬ 
mately  90  percent  of  the  total  range  of  prediction  variation.  In  addition, 
the  study  also  showed  that  the  part  grade  factor  can  affect  the  combined 
effects  of  both  temperature  and  stress  in  a  prediction.  That  is,  the  use 
of  the  upper  grads  parts  -  even  under  the  worst-case  conditions  of  stress 
and  temperature  -  produces  a  lower  predicted  equipment  failure  rate  than 
is  achieved  using  the  lower  grade  parts  under  the  best-case  conditions  of 
stress  and  temperature. 

At  the  equipment  level,  the  difference  in  predicted  failure  rates  between 
lower  and  upper  grade  parts  was  observed  to  be  approximately  ten  to  one, 
over  the  range  of  temperature  and  stress  Values  studied. 

The  distribution  of  observed  failure  rates  for  the  34  equipments  was 
examined  with  respect  to  the  total  range  of  variability  in  predicted 
failure  rates  due  to  temperature,  stress,  and  part  grade  for  each  equip¬ 
ment.  This  comparison  indicated  that,  for  the  sample  studied,  the  predic¬ 
tion  technique  is  too  sensitive  (i.e.,  predicts  too  high  a  failure  rate) 
to  the  effects  of  temperature,  stress,  and  part  grade  for  all  digital 
equipment  and  most  analog  equipment.  This  conclusion  is  based  upon  the 
following  observations: 

o  Only  19  equipments  had  observ.d  failure  rates  within  the  range  of 

prediction  variability.  All  but  one  of  these  fell  near  the  lower  limit 
of  the  range. 
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o  Almost  all  observed  failure  rates  {30  out  of  34)  fell  either  near,  or 
below,  the  lower  limit  of  the  range. 

o  Only  the  analog  equipment  (four  of  19)  shewed  observed  failure  rates 
near,  or  above,  the  upper  range  of  predicted  failure  rates. 

The  sensitivity  of  individual  equipment  failure  rates  to  part  quality, 
stress,  and  temperature  was  markedly  different  between  equipments  within 
both  the  analog  and  digital  design  approach  categories.  However,  the  sen¬ 
sitivity  to  part  quality  generally  increased  as  temperature  and  stress 
values  increased. 


c.  Details 


This  study  exercised  the  latest  RADC  stress  analysis  prediction  technique 
(RADC  Reliability  Notebook,  Volume  II)  on  34  ground  electronic  equipments 
to  develop  a  range  of  reliability  predictions  (prediction  variability 
envelopes)  for  selected  levels  of  stress,  temperature,  and  part  quality 
grade.  These  levels  were  selected  to  represent  a  range  of  temperature  and 
stress  values  extending  beyond  the  limits  of  most  ground  electronic  equip¬ 
ment  design  criteria. 


The  prediction  variability  envelopes  were  constructed  from  predicted  equip¬ 
ment  failure  rates  obtained  for  seven  points  on  the  applicable  part  failure 
rate  temperatjre/stress  curves.  Upper  and  lower  grade  parts,  per  RADC 
Volume  II,  were  represented  in  separate  prediction  envelopes.  The  seven 
points  used  tc  develop  the  failure  rates  are: 


o  0.1  Stress,  60°C  Temperature 
o  0.3  Stress,  60°C  Temperature 
o  0.6  Stress,  60°C  Temperature 
o  0.3  Stress,  40°C  Temperature 
o  0.1  Stress,  20°C  Temperature 
o  0.3  Stress,  20° C  Temperature 
o  0.6  Stress,  20°C  Temperature 

The  i£  and  values  for  a  fixed  ground  environment  were  utilized  for  each 
part.  The  values  used  for  the  longevity  factor  and  nS2  (secondary  electri¬ 
cal  stress)  were  1.0  in  all  cases.  The  above  data  were  processed  by  a 
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computer  program  (PRG-1,  Appendix  II)  which  utilized  the  following  equa¬ 
tion  to  cotain  the  prediction  variability  envelopes: 

N 

Xj  =  EQ(xb  lip  .  +  Ep  ,  (17) 


where. 

Ay  =  the  failure  rate  prediction  for  an  equipment  at  a  specified 
temperature/stress  level, 

N  =  the  number  of  part  types, 

Q  a  the  quantity  of  a  part  type, 

A,  =  the  basic  failure  rate  at  the  selected  temperature/stress 
D  level, 

lip  =  the  product  of  the  application  factors,  excluding  n^, 

11^  =  an  environmental  factor,  and 

Ec  =  an  additional  environmental  factor. 


The  equipment  level  failure  rate  data  outputs  from  the  computer  program 
were  used  to  plot  the  prediction  variability  envelopes  for  each  equip¬ 
ment.  Since  the  shapes  of  the  envelopes  were  essentially  the  same  for 
all  34  equipments,  individual  charts  are  not  included.  A  typical  ex¬ 
ample  of  the  prediction  variability  envelopes  is  shown  in  Figure  6, 
"Prediction  Variability  Envelope  (Typical)." 

To  evaluate  the  effects  of  temperature,  stress,  and  part  quality,  the 
total  variability  for  each  equipment  was  established  (i.e.,  the  difference 
between  the  failure  rate  for  upper  grade  parts  at  20°C  and  0.1  stress  and 
the  failure  rate  for  lower  grade  parts  at  60°C  and  0.6  stress).  The  total 
variability  for  each  equipment  was  used  as  an  index  of  variability  for 
that  equipment.  That  is,  the  total  variability  was  considered  to  include 
100  percent  of  the  prediction  variability  inherent  to  that  equipment  due 
to  stress,  temperature,  and  Dart  quality.  The  index  was  used  to  measure 
the  change,  in  percent,  due  to  a  change  in  part  quality  at  selected  levels 
of  temperature  (20°C,  40°C,  60°C)  and  stress  (0,1,  0.3,  0.6).  For  example, 
for  equipment  No.  2  at  20°C  temperature  and  0.1  stress,  the  difference  in 
failure  rates  between  upper  grade  parts  and  lower  grade  parts  is  83.2, 
and  the  total  variability  is  1382.0;  therefore,  the  percent  of  failure 
rate  change  due  to  part  quality  for  equipment  No.  2  is  6.0.  These  calcu¬ 
lations  were  performed  for  all  34  equipments  and  these  data  are  shown  in 
TablesLVIII  and  LIX  ,  "Percent  of  Failure  Rate  Change  Due  to  Part  Quality" 
for  digital  and  analog  equipments  respectively. 
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Stress 


Figure  6.  Prediction  Variability  Envelope  (Typical) 
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Tables  like  the  forec-oing  were  prepared  for  stress  and  temperature 
failure  rate  changes.  From  t^ese  tables  it  was  noted  that  at  the  u|»per 
limit  (i.e. ,  0.6  stress  and  oO=C  to^erature),  part  quality  could  account 
for  approxiaatlyOO  percent  of  the  total  inherent  variability  of  the 
prediction. 

The  observed  reliability  values  for  all  the  equipments  were  compared  with 
their  prediction  variability  envelopes  to  determine  if  the  variability 
inherent  in  the  prediction  technique  could  account  for  the  inaccuracy  of 
the  predictions.  Tables  LI  and  LX.I  "Variability  Envelope  Extremes 
Versus  Observed  Failure  Rates,*  for  analog  and  digital  equipments, 
respectively,  were  made  for  comparison  of  the  observed  data  with  the  pre¬ 
diction  variability  envelopes.  These  tables  illustrate  the  equipments' 
predicted  failure  rates  at  the  lowest  and  highest  points  on  the  envelopes 
(i.e.,  0.1  stress,  203C  temperature,  upper  grade  parts  and  0.6  stress, 
6G8C  temperature,  lower  grade  i  *ts,  respectively)  and  the  observed  fail¬ 
ure  rates  for  each  equipment. 

The  tables  show  that  for  the  analog  equipments,  8  cut  of  20  fell  outside 
the  prediction  variability  envelope  and  for  the  digital  equipment,  7  out 
of  14  fell  outside  the  prediction  variability  envelopes. 


The  prediction  variability  limits  were  established  for  upper  and  lower 
grade  parts,  analog  and  digital  equipments.  The  PVL's  were  established, 
in  each  case,  as  follows: 

o  The  upper  PYL  is  the  saxiEUR1  quotient  of  the  ratios  of  raxiaus 
predicted  failure  rate  to  ridpcint  predicted  failure  rate. 

o  The  lower  PYL  is  the  minimum  quotient  of  the  ratios  cf  minimum  pre¬ 
dicted  failure  rate  to  aidpoint  predicted  failure  rate. 

As  shown  in  Figure  7,  'Prediction  Variability  Limits,  Digital  Equip¬ 
ments,*  the  PVL's  for  both  upper  and  lower  grade  parts  are  C.4  for  the 
lower  limit  and  7.1  for  the  upper  limit.  The  ratios  of  predicted  failure 
rates  (upper  and  lower  grade  parts)  to  the  observed  failure  rates  for  all 
equipments  were  plotted  to  ascertain  if  the  variability  of  the  prediction 
technique  could  acccut  for  inaccuracies  tfn'ch  exist  between  the  predicted 
and  observed  values.  For  the  upper  grade  part  predictions  on  digital 
equipment,  eleven  of  the  fourteen  ratio  values  lay  within  the  PVL's;  for 
the  lower  grade  parts,  two  of  the  fourteen  values  lay  within  the  PVL's. 

Figure  8,  “Prediction  Variability  Limits,  Analog  Equipments ,K  is  a 
plot  of  the  PVL's  and  the  ratios  of  predicted  failu’-e  rates  to  observed 
failure  rates  for  the  20  analog  equipments.  The  upper  grade  PVL’s  are 
0.5  and  6.6,  lower  and  upper  respectively.  The  lower  grade  PVL’s  are  0.5 
and  6.7,  lower  and  upper,  respectively.  For  the  upper  grade  part  predic¬ 
tions,  ten  out  of  twenty  of  the  ratio  values  lay  within  the  PVL's,  and  for 
the  lower  grade  part  predictions,  seven  out  of  twenty  of  the  ratio  values 
lay  within  the  PVL's. 
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TABLE  LXJ  VARIABILITY  ENVELOPE  EXTRDCS  VERSUS  OBSERVED 
FAILURE  RATES  -  DIGITAL  EQUIPMENTS 


i  20°^  U6  I  0.6S,  60°C,  LF 


!  (S/1000  Hours)  |  (5/1000  Hours) 


891.8 

395.6 

1005.3 

151.5 


Ratio,  Predicted  Failure  Rate  to  Observed  Failure  Rate 
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on  Variability  Limits 
lalog  Equipments 


5.  SPECIAL  STUCY  #0.  4;  “PROGW#  FACTOR  RATING  STUDIES" 

8.  Objectives 

The  rating  studies  were  designed  to  determine  if  statistical  relationships 
could  be  established  between  prediction  accuracy,  using  the  RADC  Relia¬ 
bility  Notebook,  Voluae  II,  stress  analysis  prediction  method,  and  program 
factors  trtilch  are  likely  to  influence  the  level  of  achieved  reliability. 
The  major  objectives  of  this  special  study  were: 

o  Determine  if  statistical  correlations  exist  between  prediction 
accuracy  and  the  quantitative  ratings  of  the  program  factors. 

o  identify  program  factors  which  are  significant  to  reliability 
achievement. 

o  Establish  modifiers  for  use  in  prediction  models,  based  on  the 
rating  criteria  and  established  coefficients,  to  increase  predic¬ 
tion  accuracy. 

o  Obtain  information  for  use  in  other  areas  of  this  study. 

b.  Staanry 

The  program  factor  rating  studies  developed  the  use  of  a  program  rating 
grade  which  can  be  used  with  the  RADC  Reliability  Notebook,  Volume  II, 
stress  analysis  prediction  to  ascertain  the  expected  MTBF  that  will  be 
observed  on  a  particular  program  with  a  particular  rating  grade. 

c.  Details 


A  set  of  50  program-related  factors  which  were  belieyed  to  be  significant 
to  reliability  was  developed  and  grouped  into  eight  categories.  These  are 
shown  in  Tables  LXII  through  LXIX.  A  rating  scale,  based  on  the  rela¬ 
tive  strength  of  the  factors,  was  employed  to  quantify  the  strength  of 
each  factor  (e.g.,  strong  =  5,  average  =  3,  and  weak  -  1). 

The  programs  utilized  in  the  rating  studies  were  chosen  on  the  basis  of 
available  knowledge  concerning  the  50  program  factors,  availability  of 
observed  reliability  data  and  availability  of  parts  information  suffi¬ 
cient  for  making  a  stress  analysis  prediction.  All  equipments  for  each 
program  were  rated  for  each  of  the  program  factors.  Individual  equip¬ 
ment  characteristics  were  not  permitted  to  influence  the  ratings  since 
the  primary  purpose  of  the  study  was  to  establish  whether  or  not  pro¬ 
gram-related  factors  influence  the  level  of  achieved  reliability.  There 
were  three  programs,  containing  thirty  equipments,  with  sufficient  in¬ 
formation  for  rating  evaluation.  Table  LXX,  "Program  Reliability 
Measurements"  shows  the  program  rating  grades  established,  range  of  dis¬ 
persion  and  the  mean  of  the  ratios  of  observed  MTBF  to  RADC,  Volume  II, 
predicted  M18F  (i,  e.,  the  prediction  accuracy  ratios)  for  each  of  the 
three  programs. 
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TABLE  LXII  PROGRAM- RELATE!  FACTORS: 


1  ■  Degree  of  Standard  |  Parts  of  stand era,  proven  design  a r*  selected. 
Farts  Usage  I 


?  Complexity  I  Host  parts  require  a  single  action-reaction  to  provide  a  desired  1  3, 

I  single  function,  such  as  voltage  build-up  on  a  capacitor  in  8  31 

I  response  to  a  charging  current.  1 


3*  Difficulty  of  Functionl  Parts  encompassing  routinely  achievable  functions  are  selected.  <5, 

1  e: 

I  &< 


k.  Sensitivity 


Part  functions  are  achieved  by  a  relatively  wide  range  of 
stimuli,  such  as  a  wide  range  of  currents  to  actuate  a  relay. 


S 

1 


<J5  P  & 


L 

PTE#  FACTORS:  PARTS  MIX  AND  SENSITIVITY 

jlSTATiVE  CATEGORY 

knnti 

WEAK 

S ame  parts  ai-e  relatively  new  rereions  of  standard  items. 

The  new  versions  provide  enhanced  performance  of  selected 
parameters,  such  as  greater  gain  or  lower  leakage. 

3oa>e  parts  utilise  new  and 
unproven  design  concepts. 

tired 

3a»e  parte  require  complex  action  to  provide  a  function, 
such  ms  switching  through  a  stepping  switch. 

Many  complex  parts. 

*<S. 

Sane  parts,  such  as  potentiometers  and  connectors, 
encompass  functions  which  are  relatively  difficult  to 
achieve  (electrical  continuity  through  low  pressure 
contacts) 

Many  parts  encompass  functions 
which  are  difficult  to  achieve. 

r. 

Some  parts,  such  as  microwave  elements,  operate  at 
lew  signal  levels. 

Many  parts  operate  at  low 
signal  levels. 
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TABLE  LXIII  PROGRAM-RELAT 


m 

VARIABLE 

STUNS  | 

1.  Worst  Case  Analysis 

Analysis  methods  (fCKP,  etc.)  indicate  that  circuits  iriU 
function  with  worst  case  combinations  of  part  parameters. 

n 

2.  Monte  Carlo  Analysis 

Permis sable  circuit  outputs  (high  and  low)  encompass  +  L 
signas  of  the  calculated  output  distribution. 

! 

1.  Humber  of  Adjustments 

Use  of  trimmer  circuit  elements  (potentiometers,  etc.)  is 
avoided.  Temperature  compensating  circuitry  and  feedback 
(stabilization)  loops  are  employed. 

U .  TeGt  Points 


Test  points- provide  for  monitoring  of  all  critical  circuitry . 


TABLE  LOT 


PfeOGWMELXTED  FACl 


VAIIA1LES 


mumiii  cates 


slftctricfcl 


i fcrst  ease  electrical  stresses  are  less  taca  50£  cf  ratlings 


a 


Tibracicn  SBayirg 


Vsirst  -riferelicn  at  pert  lerrel  is  less  42sea  pert  rat^ags- 


Iberaal  Design 


Ibcraai  fA-ths  crcTiie  fcr  lot  teaperat&ge  rise  at  pert 
levels  eras  crier  aacsi  rase  cccdStioB*. 


a 


Ssaisiiy  Fretectien 


Bse  of  seals  arri  costings  prevent  aeastare  ccsdexaetioD  oc 
sensitise  certs  area  crder  verst  case  eccdicicc®- 


Se 

©a 


Ktf  ••'"'■41 


*  3. 
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TABU  LIY  PR06M*- RELATED  FACTORS:  0 


VAI1AILES 


fiUAllTATI! 

STIIII  j 


Procedures 


2.  JcTircnBessls 

3.  serdrre  Lerels 

A.  Icrlr  ineartai 


5.  Input/Outpot 

Parameters 


ProctXna,  detail  data  recording,  fiaclags  Cictared  into  higher 
tier  tests  u  Nil  **  design.  Ffcr— ten  to  be  tested  ere  de¬ 
termined  in  put  by  piper  amlysie  ictlrities  such  u  writ  csee 
analysis. 


Informal  p 
teet  perae. 


Tests  to  ell  eenri-i  neer»s  Judged  to  be  critical  to  system 
perf.3rer.ee. 

Ingineerlng  model  threagfc  pre-prodpctlon,  prototype  bsrdutre. 


Greeter  tin  spec,  require— rts  with  assurance  tbit  individual 
ccepcoent  limit* tiaras  are  not  nciekd  far  *11  significant  en- 
rlraasants. 


Multiple  Triable*  measured  tod  recorded,  output  parameters 

measured  mitfc  mini mm,  sudn,  and  nominal  inputs  applied,  *od 
output  lord  Triitiocs. 


Teet  serin 
psrs— ters 

Selected  p 


WsTxefi  ip 
rircr— mt-sJ 


Attribute  t 
tolerance  r 
tasting 


6-  Production  Teetirg 


1£C  %  of  sab-*3»«eblies  tested,  or  lot  samples  selected  to  an 
equivalent  lerrel  cf  confidence  for  production  units  throughout 
the  period  of  manufacture . 


Sampling,  c 
provides  &r 


nsr-dme  Lerel 


All  levels,  comprehensive  test  characteristics  at  each  lerel 


All  sysi 

at  each 


Page  137/138 


YAJtlAIH 


mimim  cate 


Oontrcl  of  Produc- 
tiee  Souipaect, 
Tools,  sad 
ProeeMM 


All  pro&tctlao  processes,  tooling,  Jigs,  fixtures,  and 
■srMnery  bsve  proven  capabilities,  demonstrate  repeat¬ 
ability,  and  sre  subject  to  sclent ifical ly  established 
process  control*,  calibration,  and  maintenance 
schedules.  All  changes  In  processes  and  equipment  sre 
carefully  proven  oat  prior  to  break -In  to  production 
and  closely  aonltored  after  break -in.  Process  and 
equipment  capability  studies  are  conducted  and  statis¬ 
tical/ engine*  ring  analyses  sre  continually  performed 
to  ensure  and  improve  process  capability  and  assure 
reliability.  Periodic  calibration  and  inspections  sre 
performed .  Responsibility  and  authority  for  process 
control  sre  firidy  established. 


Processes  sad  squlpaen 
capability,  but  change 
closely  controlled.  S 
after-the-fact,  to  corr 
production  process.  B 
consideration  in  proce 
are  established  by  the 
baaed  on  experience  to 
process  control  are  dl 
quality,  design,  and  p 


Control  of  Produc¬ 
tion  Operator* 


Operators  are  skilled,  veil-trained,  have  above- 
average  understanding  of  wort  instructions  and  work¬ 
manship  standards.  Plant  working  conditions  and 
facilities  sire  designed  to  provide  optimum  operator 
performance.  Work  instructions  sre  explicit,  current, 
and  provided  vitb  each  order.  Quality  and  quantity  of 
work  are  traceable  to  the  individual  production  opera¬ 
tor  and  strong  emphasis  is  placed  by  Management  on 
Maintaining  a  high  level  of  individual  operator  perfor- 


Operator  training,  akt 
standards  are  average, 
facilities  are  average 
are  not  usually  traces 
Management  places  a  ax> 
performance. 


FACTORS:  PRODUCTION  RELIABILITY  CONTROL 


Till  SATEfittY 


mm  and  •fulpasnt  as*  georraliy  of  prjven 
lllty,  bat  changes  art  no*  full  evaluated  or 
Ly  controlled.  Soxsal  controls  are  established, 
■^be-faet,  to  correct  problem  arising  fro*  the 
eticn  process*  Reliability  Is  not  a  significant 
Sezmtlon  In  process  control.  Fermi  controls 
rtabllshed  by  tbs  Quality  Control  organisation, 
on  experience  to  date.  Responsibilities  for 
M  control  are  divided  between  Manufacturing, 
fcy,  design,  and  production  engineering. 


lo  adequate  process  control  prograo  exists.  All 
controls  center  on  the  product.  Operators  nay 
supply  their  own  tools.  Calibration  and  service 
cycles  are  infsrmlly  established.  Changes  in 
processes  are  incorporated  oc  an  uncontrolled 
basis. 


(tor  training,  skill,  and  knowledge  of  vorknanshlp 
lards  are  average.  Working  conditions  and 
[itiss  are  average.  Quality  and  quantity  of  work 
jot  usually  traceable  to  the  individual  operator, 
basest  places  a  noderate  eophasls  on  individual 
jrmace. 


Belov -average  operator  training  and  skill  levels. 
Working  conditions  and  facilities  contribute  to 
a  reduction  in  the  quality  or  quantity  of  work 
performed  by  operators.  Management  does  not 
Motivate  operators  to  stress  quality  of  work  for 
the  individual  and  quality  of  work  is  seldom 
traceable  to  the  operator. 
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TABLE  LXVI  (CONTINUED)  PROGRAM-RELATED  FACK 


I 


VARIABLES 


QUAUTA 


STROMS 


3.  Control  of  Incasing  Inspection  and  teat  of  incoming  parts  and  materials 
Parte  and  Materials  are  accomplished  in  accordance  with  current  formal 

instructions  and  controlled  equipment*  These  instruc¬ 
tions  focus  attention  upon  the  critical  characteristics 
which  are  significant  to  the  reliability  and  quality 
of  the  item  during  the  production  and  use  cycles,  as 
established  thru  engineering  analysis.  Most  features 
of  the  Q.C.  program  in  this  area  are  at  optimum  levels 
for  reliability  purposes.  Laboratory  analyses  are 
performed  periodically  to  verify  the  conformance  of 
materials . 


am 


Inspection  and  tea 
using  a  mixed  syst 
and/or  the  dlscrel 
acteristlcs  inspe< 
to  the  reliability 
production  and  use 
program  in  this  ai 
levels  for  reliabj 


t.  Control  of  Inspection  and  test  operations  are  established  at  all 

Product  Quality  strategic  points  in  the  production  process.  They 

exercise  complete  control  of  all  production  factors 
significant  to  product  quality  and  reliability. 
Characteristics  to  be  Inspected  and  tested,  related 
procedures,  tools,  and  equipment  are  selected,  deve¬ 
loped  and  controlled  within  a  unified  quality  program 
which  is  based  upon  a  detailed  engineering  knowledge, 
analysis,  and  evaluation  of  the  sensitivity  of  the  pro¬ 
duct  to  the  production  process, and  all  applicable 
product  specifications  and  requirements .  Complete 
program  documentation,  records,  and  procedures  exist  and 
arc  kept  current.  Effective  change  control  procedures 
are  enforced.  Statistical  analyses  of  quality  trends, 
defect  rates  and  distributions  of  defect  types  for  each 
product  type,  production  department,  and  inspection 
operation  are  continually  performed  with  feedback  to 
cognizant  manufacturing  and  engineering  areas  to  initi¬ 
ate  corrective/preventive  action.  Failure  analyses 
are  performed  and  heavy  emphasis  is  placed  on  defect 
prevention  and  optimization  of  the  total  Q.C.  program. 


Inspection  and  te 
insure  that  predu 
by  the  production 
the  contract  and 
basis  for  select! 
tested  and  inspec 
are  generally  ade 
is  not  incorporate 
planning  or  opera' 


M 
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ttLATFD  FACTORS:  PRODUCTION  RELIABILITY  CONTROL 


QUALITATIVE  CATEGORY 


WEAK 


Inspection  and  test  operation*  have  &  United 
scope  and  employ  procedures  and  records  which 
are  generally  inadequate  to  insure  the  relia¬ 
bility  and  quality  of  the  item  during  the 
production  and  use  cycle.  Some  features  of 
the  Q.C.  prograa  in  this  area  exhibit  signifi¬ 
cant  deficiencies. 


Halted  inspection  and  test  operations  are  per¬ 
forated  to  achieve  a  minimum  (or  lesser)  compli¬ 
ance  with  contractual  quality  control  require¬ 
ments.  The  quality  program  is  "inspection- 
oriented,”  with  little  or  no  emphasis  on  defect 
prevention  and  no  adequate  recognition  of  the 
need  for  quality  engineering. 


TABLE  LXVi  (CONCLUDED)  PROGRAM-RELATED  FACTO! 


VARIABLES 


GUAUTAT 


5-  Lot  Control 


A  comprehensive  system  of  product  identification  or 
narking  is  established  to  accomplish  traceability  of 
production  units  in  a  unique  Banner  to  reflect  all 
significant  factors  in  the  production  process  (e.g., 
order  of  production,  production  line,  naterial  lot, 
process  changes,  date  of  production,  serial  nunber,  etc 
The  system  extends  to  all  significant  indenture  levels. 


6.  Handling,  Storage 
and  Packaging 


Material  Is  carefully  handled  from  points  of  receiving 
Inspection  through  final  packaging.  Material  is  pro¬ 
tected  from  damage  hy  well-designed  conveyors.  Jigs, 
containers,  fixtures,  etc.  Storage  conditions  prevent 
contamination,  deterioration,  and  damage.  Packaging  for 
storage  and  shipment  insures  serviceability  and  preser¬ 
vation.  Standards  end  procedures  to  insure  full 
coapllance  with  all  safeguards  necessary  to  prevent 
damage  or  deterioration  are  enforced,  and  verified  by 
tests  and  engineering  analyses. 


7.  Material  Review 


Procedures  and  practices  insure  that  all  defective 
material  is  piomptly  identified,  segregated,  and  that 
disposition  is  accomplished  in  accordance  with  review 
procedures.  These  provide  for  escalation  of  analyses 
and  corrective  action  in  response  to  the  significance 
of  the  defect(s).  Procedures  for  processing  and 
controlling  variations  are  effective,  and  docu¬ 
mentation  is  current  and  complete.  The  backlog  if 
corrective  action,  and  the  quantity  and  type  of 
deficiencies,  give  evidence  of  on  optimum  system 
which  insures  folic  /-up  on  all  major  problems. 


RELATED  FACTORS:  PRODUCTION  RELIABILITY  CONTROL 


mnxm  category 


A VEKA! £ 


xtoctloa  units  are  serialized,  date  cooed -or 
lociated  paperwork  io  identified -under  a  limited 
item  of  lot  control  vhich  does  not  include  all 
jnificant  factor*  in  the  production  process  (e.g. . 
}ot  process  changes  are  not  reflected)  and  does  not 
rlude  all  significant  Indenture  levels. 


iterial  handling;  storage;  and  packaging  standard* 
id  procedures  equal  or  exceed  good  co— erciai  prac- 
Lces,  and  usually  comply  vlth  the  applicable  KIL-aTD's 
waked  by  contract.  Design  of  production  handling 
juipment  and  containers,  and  handling  practices  of 
rodwcticm  workers  occasionally  present  some  hazards 
>  equipment.  Little  or  no  verification  of  the 
rfectiveness  of  these  standards  and  procedures  is 
icompliahed. 


Sene  rally  good  control  over  defective  material  is 
accomplished,  using  adequate  procedures  and  standard 
practices,  engineering  review,  corrective  action, 
and  disposition  instructions  are  usually  good,  but 
do  not  always  escalate  in  response  to  the  magnitude 
or  potential  impact  of  the  deficiency.  Records  are 
adequate,  repetitive  deficiencies  and  a  bajklog  of 
incomplete  corrective  action  exist.  There  is  not 
an  adequate  system  to  insure  follow-up  on  all  major 
problem  areas. 


Serialization  or  lot  identification  may  exist, 
but  is  sot  meaningful  or  useful  in  most  cases, 
(e.g.,  lot  rises  are  too  large,  identification 
is  not  changed  to  reflect  significant  varia¬ 
tions  in  processes,  materials,  etc.). 


Material  Is  not  carefully  handled  at  some 
points  in  the  production  process.  Production 
handling  equipment.  Jigs,  storage  facilities, 
p^oir«£rtng  methods,  and/or  containers  subject 
the  equipment  to  adverse  conditions  or  hazards 
to  an  extent  vhich  reflects  significant 
deficiencies  in  design,  planning,  or  management 


Procedures  and  practices  loosely  control 
defective  material.  They  do  not  insure  that 
the  disposition,  analysis,  and  corrective 
action  will  be  commensurate  with  the  signi¬ 
ficance  of  the  defect.  Documentation  review 
gives  evidence  of  an  inadequate  system.  There 
is  no  evidence  of  follow -up  on  mejor  problems. 
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TABLE  LXVII  PROGRAM- RELATED  FACTORS:  DESIGN 


VARIABLES 

QUALITATIVE 

CATEQOl^ 

STROM 

1.  Simplicity  of 

Relatively  few  building  blocks  (amplifying  stages. 

A  fa 

Design  Logic 

electromechanical  items,  etc.)  are  employed  to 

conf: 

achieve  required  functions. 

isti< 

2.  Circuit  Sensitivity 

Significant  circuit  designs  will  tolerate  failure  or 

Sign: 

to  Part  Failure/ 

degradation  of  principal  parts  (e.g.  through  built-in 

degn 

Degradation 

redundancy  or  added  capacity  at  the  part  level)  without 

significant  failure  or  degradation  of  circuit  performance. 

3 .  Unit  Sensitivity 

Significant  unit  designs  will  tolerate  failure  or 

Signi 

to  Circuit  Failure/ 

degradation  of  principal  circuits  (e.g.  Through  built-in 

degrc 

Degradation 

redundancy  or  safety  margin  on  input/output  values) 

without  loss  of  primary  fi. action. 

k .  System  Sensitivity 

Ho  limited-life  items  are  used  in  any  function  necessary 

to  Degradation  ani 

for  system  performance. 

gy -gtj 

Wearout 

5.  Design  vs  State  of 

All  functions  'ire  easily  achieved  using  established,  proven 

Tne  Art  (SOA) 

technology  throughout  the  design,  with  capability  and 

•vo  v 

tnifw 

perforssnfce  well  above  standards. 

VU*  A  ’ 

from 

use  ! 

FACTORS:  DESIGN  FEATURES  AND  DEGRADATION 


A  feu  hrfliHwg  blocks  are  added  to  the  basic  (simple; 
configuration  to  etdmfw*  selected  perforate:*  character¬ 
istics,  oar  to  add  supplementary  functions. 


Say  "eititj1  are  added. 


Significant  circuit-  designs  will  tolerate  failure  or 
degradation  of  principal  parts  to  a  lisdted  extent. 


So  attearpC  to  eonpenssie 
for  part  failure  or 
degradation  in  any 
design- 


Significant  unit  designs  will  tolerate  failure  or 
degradation  of  principal  parts  to  a  United  extent. 


So  stteapt  to  ccnpenszte 
for  past  failure  or 
degradation  in  any  unit 
design - 


Halted  life  Item  any  does  not  sipilficautly  affect 
system  reliability,  but  »y  cause  same  falls ras. 


Halted  life  lteam  are  a 
principal  source  at  system 
failure. 


Moat  functions  are  readily  acfcieveeble  within  the 
currant  3QA,  but  acme  designs  are  close  to  margins! 
from  •  parfonmtnce  standpoint,  or  Involve  limited 
turn  of  new  technology  sot  yet  proven  in  service. 


Very  difficult  for  the 
design  to  achieve  all 
functions  within  SQA. 
fcv,  untried  tede>ology 
nay  be  used  extensively. 


TfiBLE  Ofill  ?w Gm-mmsss  fm 

l 

I 

t 


?arts 

Sraadaraincrico 


6. 


5 pec  i  l  icaticcs 


Score*  Selectice 


QssJLificatfca  Testing] 
iSocre*  or  Procsriag 
fcliTjw) 


ScrMai^-'&ra-ja/H 

?rogr.Kt 

Acceptance  Tearing 
(Source  or  Procuring 
Agency) 


Tie  procsriag  ytiritj  «dit«3a  a  strong 
cifgiiarfcr.  sfcJxi  txerdtts  cecrel  over  pests 

specification,  srarafares.  proeaxeaest,  <csaH££catiatB, 
acceptance,  senre*  selection  sa£  application.  Parts  specialists 
regularly  oesd:  virh  designers  o®  parts  application,  and  scs4 
closely  vico  pcoocrenest.  Reality  engineering.  reliability,  axd 
proioctsc®  co  iassre  cytiaaat  pans  — ay  ~  Tonal  pcocecsrex 
effectively  liadt  /eswrrci  tie  use  ef  son-steaiard  parts. 

Procuring  Agency  establishes  ao£  coatrcls  past  an*  test 
specif icsticas  in  excess  of  HE.  ce^aireaest*  emcU -carfi 


ly  xa  Itipertixl  scientific  process  afeieh  iarxres  selective  of 
a  reliable  source. 

A  coepres  ensive  ^tarl  f  ficatioo  test  pcopa*.  npoziif  proof 
of  part  acceptability,  is  provided  sick  respect  to  all 
aissico  eoriroaargrs.  feriaiic  retesting  is  accaaplished. 
Test  levels  ace  frequently  Ugbcr  than  slssioa  repd  reseats  - 

?arts  screened  for  pameter  drift  or  subject  to  1S3I 
conditioning  or  boro-in  prior  to  acceptance  sod  assesbly. 


Parts  are  tested  1002  vitk  respect  to  uuscroas  paraweters  at 
levels  vtiica  exceed  XH.-SP3C  rewiraects. 


Par 


to 


HU 


a* 

(e.; 

Qaa 

lee 


Pan 


Aco 


Established  Parts 
Palitbiiity 


If  eart  reliability  is  not  established  by  use  of  *31  Specs  Uses 
rd table  dtii  la  large  sanples  are  used  to  denocstrate  part 
failure  rates. 


•  U.r) 


Tmtxm  ot  njDtej  yew  foi|s  arc  ywiily  teiirtri. 
Mp  rr— itrfu  exist  Ss*e  tie  iaaisltoi  teiptr 
wulws  laiftaint  co«tr>I  ewer  ysrts  telKtte,  sttjtct 
tv  IMU«  nmillssoe  iy  ett.r  esfltsrfag  toeifliaa. 


So  effect-ice  standards  orif 
wtrsiiwi,  TTSfixt,  or 
itcwntiriae  exists. 


Military 


test  nptoettts  are  seed. 


Tkwt  syeciffcat&te  sheets  cr 
catalogs  are  ase£. 


If  «  poem  «Udi  wr>fer  Sc  see  ala*ays  tsercs 
(t.g.  qPL  listing^  reliability  of  sssm. 


selects  seance  I 

-Kmtioct  relia&iilty  caBsrxaistJ 


Ij—HfrTlnw  tesrlm  jer  SU.-5S-2RI  is  accsyUsM. 
is  occasionally  fctfaaei. 


Qaallfira  trior  testing,  yer  3SI 
STD-202,  acr  ie  actaqilisxei. 
Set  Is  less  tian  socyate. 


2trt  ixt-ia  Is  scanyl^jM  teiag  awnAly  cr  test. 


tests,  yer  xilitary  sydficatlees,  aaS 
*■?!*  tests  of  critical  yaraaerexx. 


Se  samitg  or  Somo-in. 


Acte? raaoe  tests  are  frecoesiljl 
csitttf.  I 
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TABLE  LXHII  (CONCLUDED)  MOGMH-iEUIEO 


QUAUTATiK  CATEG 


VAISAIIES 


STMIt 


?art*  I* 

Jadjsls 


(Or:  am  cfftcttvc  prognm  Mt  pwriW). 


Fdlare 


Source  Sr— filar* 


ftteniNi  tf/mcj  pcrfxac  QA  —Hfr,  *rrw«*  »  *“■«  * 
nsite£  inspector  st  tie  aotzat,  mad  mmimt mSmm  a.  seedor-rsd** 
fjstn  or  c^sivalcfit  foeAtcx  of  rest  i*t*  ao»ly*l*. 


10.  C&xa$e  Carnal 


Ttafcr  m:  scbsrf  t  use  proccriag  *jbbej  wt  sppioee  »Jor 
process  an)  &nviat  chaoses. 


-  . .5  ai 


FACTORS:  PARTS  PWGMK 


IVE  CATEfiStT 


AfEtASE 


VEAX 


Failure  «e<ri  art  identified,  evaluated,  aad  iliiaiiri 


Seeded  but  not  provided  or  ao 
put  failure  analysis  ptottac. 


Periodic  audita  per  1QL-Q-9C58A  are  perfoxued. 


So  audits,  a e  feedback  other 
than  rejection  reports. 


TTrrff  are  controlled  only  den  part  latere a  uyiM,  1  tty 
is  affected  or  vfeea  Procuring  Agency  discovers  proole*. 


So  control  over  part  vendor 


1 


TABLE  LXIX  PROGRAM-RELATED  FACTORS: 


VAKIAIIE 

QUALITATj 

SIMM 

1.  Program  Management 

Report*  to  Tiee-President 

Reports 

(Reliability  Manager) 

or  Gene  ml  Manager 

2.  Procuring  Agency 

Monthly,  with  specific  task  status  review 

Quarter! 

Program  Reriev 

3«  Reliability  Math 

Task  conducted  during: 

Task  com 

Models,  Predictions 

Pre-design 

Pre-d 

and  Estimates 

Paper  Design 

Paper 

Breadboard  Design 

Pre-Prod  Hardware 

Operational  Hardware 

^  failure  Mode  Analysis 

Comprehensive  failure  mode,  effects,  and  criticality  analysis 

Provide  : 

by  Reliability  Engineers 

ability) 

5.  Reliability  Trade-Off 

Reliability  ia  major  trade-off  factor 

Reliable 

Analysis 

6.  He liability  Design 
Acquirement* 


Reliability  department  stipulates  and  enforces  design  constraints 
in  the  areas  of  derating,  environmental  requirements,  redundancy, 
part* / mrteri al  application.  Worst  case  design,  etc. 


Reliabi: 

enforce 


QUALITATIVE  CATEGORY 


_ _ _ 

aveim  i 

WEAK 

Deports  to  Chief  Engineer 

Deports  to  Project  Engineer 
or  other 

Quarterly  with  general  task  status  review 

*o  Procuring  Agency  Review 

. 

Task  Ooodueted  during: 

Pre -design 

Piper  design 

Task  conducted  during: 

Paper  design 

St 

Provide  failure  rates  to  other  organisations  ( such  as  Maintain* 
ability)  and  review  other  organisation  output 

Ho  failure  node  analysis 

Reliability  is  minor  trade-ofC  factor 

Reliability  trade-off 
analysis  not  performed 

mint* 

dancy, 

i 

Deli ability  department  stipulates  design  constraints  but  does  not 
enforce  the* 

Ho  design  constraints 

A 

i 

s 


f 
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TABLE  LXIX  (CONCLUDED)  PROGRAM- RELATED  FAC 


QUAliTATI 


VARIABLE 


Design  Review 


8.  itest  Program 


Failure  Reporting, 
Analysis  and  Oorreet- 
ive  Action 


Reliability  department  takes  active  role  in  design  reviews  and 
nay  organise  and  chair  the  review.  Reviews  consist  of  System 
Design  Review  (pre-design)  Preliminary  Design  Review  (initial 
paper  design)  Critical  Design  Review  (final  paper  design, 
technical  Acceptance  Review  (Engineering  Model)  first  article 
Configuration  inspection  (Production) 


Re  11  ability  department  is  instrumental  in  test  planning  and  has 
approval  authority  of  test  documentation. 


Reliability  department  establishes  and  controls  program  that 
ensures  a  closed  loop  system  for  analysis  of  each  failure  event. 


Reliabil 

Critical 


to  re  11  ah 


High  perci 
conducted 


RELATED  FACTORS: 


RELIABILITY  PROGRAM 


QUALITATIVE  CATE60IY 


_id 

:n 

tl 

L* 


has 


Reliability  department  attend*  design  review  meetings ,  usually  a 
Critical  Design  Review  (initial  paper  design)  is  the  only  one  held,  j 


*0  Design  Review 


Me  reliability  effort 


So  reliability  effort 


t 

pent. 


Hgb  percent  of  failure  events  are  documented,  but  analysis  is 
conducted  only  on  known  problem  areas . 


Many  failure  events  go 
unreported.  Analysis  ia 
conducted  by  other  than 
reliability  department . 


8 


an 
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j  ABLE  LXX 


PROGRAM  RELIABILITY  MEASUR£}€KTS 


Program 

Equipment 

Code  Numbers 

Rating 

Grade 

Range  of  Dispersion 
(HTBF0/KTBFp) 

Mean 

(MTBF0/KT8Fp) 

I 

12,13,14,15,16, 

17,18,19,20,21, 

22,23 

3.3 

0.06  to  1.8 

0.66 

II 

25,26,27,28,29, 

30,31 

m 

0/5  to  10.2 

2.26 

III 

1 ,2,3,4, 5,6,7, 
8,9,10,11 

4.8 

0.8  to  26.4 

6.94 

Thu  distribution  of  the  equipment  ratios  are  shown  in  Figure  9,  10  and 
11  "Distribution  of  Prediction  Accuracy  Ratios",  (Progrjms  I,  II  and  III 
respectively).  It  is  noted  from  Table  LXX  that  the  range  of  dispersion 
is  consistent  between  programs  regardless  of  the  rating  grade,  but  the 
means  of  the  ratios  increase  as  the  program  rating  grades  increase. 

That  is,  there  is  no  correlation  between  the  precision  of  the  prediction 
for  individual  equipments  and  the  program  rating  grade  but  definite 
correlation  exists  between  the  accuracy  of  the  program  prediction  and 
the  program  rating  grade. 

The  RADC  Reliability  Notebook,  Volume  II  recognizes  the  existence  of  two 
part  quality  levels  (upper  and  lower)  but  implies  that  intermediate  levels 
may  exist  and,  in  fact,  be  continuous  from  the  very  low  level  to  the  very 
high  level.  It  appears  from  the  rating  studies,  that  a  continuous  part 
quality  grade  does  exist  and  may  be  applied  as  a  modifier  at  the  program 
level . 

To  illustrate  the  existence  of  a  program  modifier  based  on  the  rating 
grade  established  for  the  program.  Figure  12,  "Program  Rating  Grade  vs 
Prediction  Accuracy  Ratios,"  was  prepared.  This  figure  was  formed  by 
plotting  the  program  means  of  the  ratios  of  observed  Ml'BF  to  predicted 
MTBF  versus  the  program  rating  grade  (i.e.,  the  mean  of  the  program  pre¬ 
diction  accuracy  ratios).  The  plotted  means  were  connected  with  a 
straight  line.  This  line  defines  the  program  mean  for  a  population  of 
programs  with  varying  rating  grades  between  the  range  of  3.3  and  4.8, 

This  will  allow  the  use  of  a  rating  grade  modifier  for  more  accurate  pro¬ 
gram  predictions  via  RADC  Volume  II. 

For  example,  given  a  program  with  a  rating  grade  of  3.8  and  an  RADC  Vol¬ 
ume  II  prediction  of  1000  hours  MTBF  for  the  program,  what  is  the  expected 
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29?^  g*™1  ^  to  predicted  KTBF  (i.e.,  the  prediction  accuracy 

EJIJ2  i^«f9U^L12,  Si  e2*cHtpredict1on  accuracy  ratio  for  the 
1 •-2hent RA0C  Vc1uae  11  Prediction  for  the  pro- 

frn^eS1^]1?  Jh  1Af8? J!T  P™11**10"  accuracy  ratio  for  the  program 
is  reduced  to  1.0,,  the  desired  va.ue.  Looking  at  this  in  another  way 

SfJfJ*  C8!l  US^tLft5nn!ne  *5at  observed  X™*  car  be  -:*r>ectfcd  in 
<*eJJJ*iJod1  . that  the  expected  prediction  accuracy  ratio 

thJ'/i ’iftWiSXl'19  ™ln  S  **  ^h£  r*t10  0f  observed  to  predicted  MTBF, 
if*?.*  I*48'  M000*  *  14S0  15  ^  expected,  observed  KTBF  in  the 

PfU  I2r*Squlp?eni  h,vinf  a/atin9  9^de  of  3.8.  The  above  results  are 
ba«d  on  the  data  fro*  only  3  programs.  However,  due  to  the  potential 
slgnlticance  of  these  results  it  is  felt  that  further  effort  in  this  area 
is  warranted. 
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Figure  9.  Distribution  of  Prediction 
Accuracy  Ratios  Program  I 
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SPECIAL  srjsr  W.  5: 


FAILURE  RATE  *E2SU$  7I>€* 


2.  Object!  ves 

The  latest  ravisior.  of  the  RADC  fteliacility  Notebook  Indicates  that  part 
failure  rates  are  not  constant  with  tiae.  This  study  was  cornJucted  to 
evaluate  the  relationships  between  failure  rates  at  the  equipment  level 
and  accumulated  operating  tiae  for  a  variety  of  ground  electronic  equip¬ 
ment.  The  study  exsnnes  failure  rates  versus  tiae  for  approximately 
forty  types  of  fixed  ground  electronic  eouipaents. 

Riots  of  failure  rate  versus  tine  are  developed  based  or.  quarterly  inter¬ 
vals  to  eliminate  short-tern  variations  introduced  by  sonthly  anomalies. 
Each  equipnent  plot  presents  three  views  of  the  data: 

o  The  actual  ni»ber  of  failures  per  10C»3  hours  per  period. 

o  The  cumulative  failure  rate  versus  tiae  for  the  total  period. 

o  The  one-year  moving  average  of  failures  per  1COO  hours.  This  moving 
average  was  used  to  reflect  tne  influence  of  later  data  ard  to  provide 
more  positive  trend  indications. 

fc.  Sanaa  ry 

The  following  conclusions  can  be  drawn  froc  the  analysis  of  failure  versus 
time  characteristics  of  the  equipment. 

o  In  72.95  of  the  equipments  analyzed,  failure  rates  are  no:  constant 
with  time.  For  the  equipment  where  the  failure  rate  is  decreasing  w*th 
time,  the  minimus  failure  rate  decrease  was  201  and  the  maximum  decrease 
was  9401.  Eighty  percent  of  these  equipment  experienced  failure  rate 
decrease  between  201  and  165*.  For  the  equipeents  where  the  failure 
rate  is  increasing  with  time,  the  equiptnents  showed  failure  rate  in¬ 
creases  of  1455,  801,  201  and  0.0*.  Equipment  C-l  had  a  failure  rate 
versus  time  pattern  tnat  was  definitely  increasing,  even  though  the 
year  to  year  estimates  'were  identical. 

o  Receiver  and  computer  type  equipment  are  mere  likely  to  exhibit  con¬ 
stant  failure  rates  versus  time  than  transmitter  and  display  equipment. 

o  Failure  rate  changes  from  year  to  year  are  greater  than  plus  or  minus 
50*  in  48.41  of  the  equipments  analyzed. 

o  A  positive  correlation  exists  between  failure  rate  versus  time  charac¬ 
teristics  and  percent  failure  rate  change. 


t  -5.-r  -  i  t»qgwic  ty 


t .  Details 

(1)  Failure  State  Versus  Tine  -  Statistical  Analysis 

Hiis  section  presents  a  discussion  of  the  failure  rate  versus  tine 
data  derived  frcrs  the  field  reports.  Toe  first  consideration  is  tc 
determine  if  the  failure  rate  is  increasing,  decreasing,  or  can  be 
considered  constant.  This  da ten»i ration  ca"  be  reached  bv  exasininc 
the  curves  to  observe  the  general  shape  and  tc  perform  statistical 
tests  en  the  data.  The  statistical  test  used  was  the  chi-square  test. 
The  fundamental  theorem  underlying  the  test  is  that:  given  2  sets 
of  observed  outcomes  and  a  latching  set  of  expected  outcomes  for 
the  possible  outcomes  of  an  experinent  that  is  performed  many  tirves; 
then  as  the  number  of  observations  become  large,  the  distribution  of 


Cc.-  -  e.)~/e. 

I  .  I 


approaches  a  chi-square  distribution  with  k-l-L  decrees  of  free  deg, 

»rf>ere: 

o  =  number  Gf  observations  {failures  per  unit  time)  per  tine 
interval  k. 

e  =  expected  number  of  observations  (failures  per  unit  tire} 
per  tine  interval  k„ 

k  =  nueber  of  tine  intervals  considered  (number  of  quarters}. 

i  =  nunber  of  parameters  estimated  froo  the  data. 

The  number  of  observations  equals  the  nurber  of  failures  per  1QD0 
hours  per  quarter.  The  expected  nusber  of  observations  ecuals  toe 
average  nue&er  of  failures  per  1000  hours  tines  the  hours  per  quarter. 

»  _  Failures  Hours  ,.  .. 

e  '  im  Hours  x  Quarter  =  Fa'  ures/Quartsr 

A  statistical  test  of  hypothesis  is  established  fer  tne  data  as 
follows: 

Hq  :  Failure  rate  is  constant  with  time 
H-|  :  Failure  rate  is  not  constant  with  time 


A  statistical  significance  level  is  established  for  the  test.  If  tne 
calculated  value  of  chi-square  exceeds  the  value  of  chi-square  in  the 
chi-square  table  at  the  desired  significance  level,  the  initial 
hypothesis  (H  )  is  rejected  and  H,  is  accepted.  In  this  analysis, 
acceptance  of  H  means  the  hypothesis  of  a  constant  failure  rate  cannot 
be  rejected.  The  acceptance  of  H,  means  that  at  least  one  of  the 
equipments  in  the  subgroup  has  a  failure  rate  that  is  not  constant 
with  time. 
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In  performing  the  chi-square  test,  the  restriction  that  the  expectation 
(expected  nuaber  of  failures)  per  cell  should  exceed  five  was  observed. 
If  this  was  not  the  case  for  a  particular  equipment,  cells  were  com- 
bined  such  that  the  requirement  was  satisfied.  Thus,  if  no  cells  were 
combined  and  since  9  quarters  were  Involved,  the  maximum 
nuaber  of  degrees  of  freedom  is  5-1 -t  =  7.  For  this  formulation  i*  1, 
as  the  expected  number  of  failures  is  estimated  from  the  data 
requiring  that  one  degree  of  freedom  be  subtracted.  If  the  number  of 
quarters  of  data  is  iess  than  9,  or  cell  combination  was  required,  the 
nuaber  of  degrees  of  freedom  was  reduced  correspondingly. 


table  lxxi 

TRANSMITTER  TYPE  EQUIPMENT  -  CHI-SQUARE  COMPARISON 


Equipment 

- - -  ■  -  ■  - 

Chi-Square  at 
95%  Confidence 

Calculated  Value 
of  Chi-Square 

T-l 

14.1 

338.5 

T-2 

14.1 

1023.2 

T-a 

14.1 

335.2 

T-4 

14.1 

670.7 

All  calculated  values  of  chi-square  exceed  the  95%  significance  value 
of  chi-square  listed  in  the  above  table.  (Seven  degrees  of  freedom 
were  used  for  all  equipments).  Tnis  means  one  or  more  of  the  units 
per  equipment  group  has  a  measured  failure  rate  that  could  not  be 
considered  constant  in  time.  In  fact,  for  the  four  equipments,  the 
chi-square  value  would  reject  the  hypothesis  at  greater  than  the 
0.995  confidence  level  (the  limit  of  the  available  table). 
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TABLE  LXXII 


RECEIVER  TYPE  EQUIPHENT  -  CHI-SQUARE  COMPARISON 


Equipment 

Chi-Square  at 
95%  Confidence 

Calculated  Value 
of  Chi-Square 

R-l 

14.1 

68.4 

R-2 

14.1 

43.6 

R-3 

6.0 

11.5 

R-4 

14.1 

26.6 

R-5 

14.1 

11.2 

R-6 

o.8 

0.1 

R-7 

— 

— 

R-8 

— 

-- 

In  four  of  the  six  cases,  the  hypothesis  of  a  constant  failure  rate 
was  rejected.  For  R-5,  one  of  the  cases  where  chi-square  was  not 
significant,  only  one  degree  of  freedom  was  available  due  to  data 
grouping  requirements.  Most  of  the  calculations  show  chi-sauare  to 
be  significant  at  greater  than  the  0.995  confidence  level.  (R-l ,  R-2, 
R-3,  R-4).  The  chi-square  test  was  not  performed  for  equipments  R-7 
and  R-8  since  the  failure  expectancy  was  not  sufficient. 


164 


TABLE  LXXIII 


COMPUTER  TYPE  EQUIPMENT  -  CHI-SQUARE  COMPARISON 


Equipnient 

C^ii  -Square  at 
95X  Confidence 

Calculated  Value 
of  Chi-Square 

C-1 

14.1 

44.7 

C-2 

14.1 

9.0 

C-3 

14.1 

84.7 

C-4 

— 

C-5 

3.8 

2.0 

C-6 

14.1 

26.1 

C-7 

14.1 

10.7 

C-8 

6.0 

32.0 

C-9 

— 

-- 

C-10 

S.O 

54.9 

c-n 

-- 

— 

C-12 

3.8 

47.6 

C-l  3 

— 

— 

C-l  4 

9.5 

9.2 

C-l  5 

a)  Electrical 

9.5 

8.0 

b)  Mechanical 

9.5 

40.9 

For  the  computer  equipment  chi-square  calculations  could  be  performed 
for  11  of  the  15  equipments.  On  the  other  four  equipments,  cell 
■erging  requirements  prohibited  use  of  the  chi-square  test.  Of  the 
equipments  tested,  7  showed  rejection  of  the  constant  failure  rate 
hypothesis;  for  the  rerainder  the  hypothesis  was  not  rejected.  For 
equipments  where  the  hypothesis  was  not  rejected,  two  were  the  large 
computers  (€-2  and  C-7),  one  was  the  electrical  components  of  the 
magnetic  tape  units  (C-15).  one  was  an  interface  equipment  (C-14) 
and  one  was  a  timing  unit  (C-5).  Four  other  interface  equipments 
and  three  other  timing  equipments  chi-square  test  resulted  in  rejec¬ 
tion  of  the  constant  failure  rate  hypothesis. 

TABLE  LXXIV 

DISPLAY  TYPE  EQUIPMENT  -  CHI-SQUARE  COMPARISON 


Equipment 

Chi-Square  at 

95  %  Confidence 

Calculated  Yalue 
of  Chi-Square 

D-l 

6.0 

15.5 

D-2 

14.1 

47.0 

D-3 

— 

— 

D-4 

6.0 

6.3 

D-5 

14.1 

48.0 

D-8 

— 

— 

D-7 

9.5 

3.1 

D-8 

6.0 

9.7 

D-9 

— 

Chi-square  tests  were  performed  on  6  of  the  S  display  equipments 
and  5  of  these  6  showed  significance. 
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(2)  Graphs  of  Failure  Rate  Versus  Time 

This  section  presents  the  plots  of  failure  rate  versus  time  for  each 
of  the  equipment  groups.  Each  ploc  presents  the  average  unit  failure 
rate  per  1000  hours  per  quarter,  (designated  by  a  dot  (.));  the  cibw- 
lative  failure  rate  versus  time,  (designated  by  a  triangle  (a));  and 
a  one-year  moving  average  failure  rate  versus  time,  (designated  by  a 
circle  (0)).  Accompanying  each  plot  is  a  discussion  of  the  data. 

In  the  discussion  for  each  equipment, the  calculated  value  of  chi- 
square  is  presented.  Also,  failure  rate  estimates  for  all  the 
equipnents  per  equipment  group  for  the  first  year,  the  second  year, 
and  overall  (aenulative  failure  rate)  are  listed.  For  example, 
referring  to  transmitter  equipment  T-l ,  each  point  per  quarter  is 
obtained  by  dividing  the  total  failures  by  the  number  of  equipments. 
(In  this  case  six  transmitters  are  included  in  the  equipment  group.) 

The  failure  rate  estimate  for  the  first  year  is  obtained  by  dividing 
the  total  operating  hours  for  the  six  transmitters  for  the  first  year 
by  the  total  number  of  failures,  and  adjusting  the  value  to  the  base 
of  failures  per  1000  hours.  A  similar  procedure  was  used  to  obtain 
the  estimate  of  failures  per  1000  hours  for  the  second  year.  The 
cunulatiYfc  or  overall  failure  rate  is  obtained  by  dividing  the  total 
number  of  hours  by  the  total  failures,  and  adjusting  to  the  failure 
rate  base  of  failures  per  1000  hours. 

Finally,  it  should  be  noted  that  on  all  charts,  the  abscissa  is  broken 
to  indicate  that  data  are  not  shown  for  the  time  interval  between  the 
equipment  IOC  (Initial  Operational  Capability)  date  and  the  beginning 
of  the  first  quarter  represented  in  the  data 


Transmitter  Equioment  T-l 


The  point  estimates  of  the  failure  rate  per  quarter  and  the  cumulative  fail¬ 
ure  rate  decrease  with  time.  The  value  of  chi-square  is  338.5,  highly  signi¬ 
ficant.  The  point  estimate  of  the  failure  rate  for  the  first  year  is  169/1000 
hours,  for  the  second  year  is  109/1000  hours,  a  decrease  of  35. 5X,  and  an 
overall  failure  rate  of  123/1000  hours  for  27  months  of  operation.  Hence, 
it  is  concluded  that  the  failure  rate  of  this  equipment  is  decreasing  with 
time.  Figure  13  plots  the  average  failure  rate  for  the  six  transmitters. 
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Transmitter  £quipn.ent  T-2 

The  point  estimate  of  the  failure  rate  per  quarter  and  the  cumulative 
failure  rate  decrease  versus  time.  The  value  of  chi-square  is  1023.?, 
highly  significant.  The  point  estimate  of  the  failure  rate  for  the  first 
year  is  240/1000  hours,  for  the  second  year  is  114/1000  hours,  a  decrease 
of  52.4%,  and  sn  overall  failure  rate  for  27  months  of  operation  is  162/1000 
hours.  Hence,  it  is  concluded  that  the  failure  rate  of  this  equipment 
is  decreasing  with  time.  Figure  14  plots  the  average  failure  rate  for  the 
ten  transmitters. 


Quarters 

Figure  14.  Failure  Rate  Versus  Time  -  Equipment  T-2 
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Transmitter  Equipment  T-3 

This  equipment  Is  similar  to  transmitter  equipment  T-l,  but  the  data  are  sub¬ 
stantially  dissimilar.  The  cumulative  failure  rate  versus  time  initially 
decreases*  reaches  a  Miniaum  at  12  months,  climbs  substantially  the  next  12 
months,  and  then  shows  a  tandency  to  decline  during  the  last  quarter. 

The  failure  rate  for  the  first  year  is  65/1000  hours,  for  the  second  year 
104/1000  hours,  with  an  overall  failure  rate  of  78/1000  hours.  Chi-square 
is  highly  significant  at  335.2.  Based  on  the- data,  at  least  one  of  the  four 
transmitters  must  have  had  a  failure  rate  that  was  increasing  with  time. 
Figure  15  shows  the  average  results  for  the  four  transmitters. 
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Fig-jr*  15.  Failure  Rate  Versus  Time  -  Equipment  T-3 
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Transmitter  Equipment  T-4 

This  hardware  is  similar  to  transmitter  equipment  T-2.  The  cumulative 
failure  rate  decreases  with  time.  The  value  of  chi-square  is  670.7,  highly 
significant.  The  point  estimate  of  the  failure  rate  for  the  first  year  is 
154/1000  hours,  for  the  second  year  is  11 5/1 0 JO  hours,  a  decrease  of  25.4%, 
and  the  overall  failure  rate  for  27  months  of  operation  is  124/1000  hours. 
Hence,  it  is  concluded  that  the  failure  rate  .for  this  equipment  is  decreas¬ 
ing  with  time.  Figure  16  plots  the  average  failure  rate  for  the  eight 
transmitters. 


Quarters 

Figure  16.  Failure  Rate  Versus  Time  -  Equipment  T-4 
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Receiver  Equipment  R-1 

The  cumulative  failure  rate  and  the  moving  average  decrease  with  time  for 
this  equipment.  The  point  estimate  of  the  failure  rate  for  the  first  year 
is  25.6/1000,  for  the  second  year  is  13/1000,  a  decrease  of  49.1%,  with  the 
overall  failure  rate  of  19.1/1000  hours  for  27  months  of  operation.  The 
value  of  chi-square  is  68.4,  highly  significant.  The  failure  rate  for 
this  equipment  is  decreasing  with  time.  Figure  17  plots  the  average  failure 
rate  for  the  18  receivers. 
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Figure  17 


Failure  Pate  Versus  Time  -  Equipment  R-l 
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Receiver  Equipment  R-2 

The  failure  rate  versus  time  plot  for  this  equipment  is  one  of  the  most 
peculiar  observed  in  the  study.  The  point  estimates  of  the  failure  rate  per 
quarter  form  an  oscillatory  pattern,  giving  rise  to  an  oscillatory  cumulative 
cailure  rate  plot.  From  this  data,  it  appears  that  the  equipment  experienced, 
on  a  yearly  basis,  some  problem  that  resulted  in  a  high  removal  rate,  and 
then  was  temporarily  repaired.  The  point  estimate  of  the  failure  rate  for 
the  first  year  is  1 2 . 5/1  COO  hours,  for  the  second  year  is  10.8/1000  hours, 
with  the  overall  failure  rate  for  27  months  of  operation  of  12.7/1000  hours. 
Thus,  on  a  yearly  basis,  the  failure  rate  was  constant.  However,  due  to  the 
unique  slope  of  Figure  18,  the  only  conclusion  that  is  reasonable  is  that 
the  failure  rate  for  these  16  equipments  is  not  constant  with  time. 
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Unit  Failure  Rate  Per  1000  Hours 


The  cumulative  failure  rate  and  the  moving  average  decrease  with  time.  The 
point  estimate  of  the  failure  rate  for  the  first  year  is  3.Q/10Q0  hours,  for 
the  second  year  is  1.1/1000  hours,  a  decrease  of  63. 4S,  wUn  an  overall 
failure  rate  for  27  months  of  operation  of  1.9/10QG  hours.  The  value  of 
chi-square  is  11.5,  highly  significant.  The  failure  rate  for  this  equip¬ 
ment  is  decreasing  with  time.  Figure  19  plots  the  average  failure  rate  for 
the  13  receivers. 


Receiver  Equipment  R-4 

This  hardware  is  similar  to  receiver  equipment  R-l .  The  cumulative  failure 
rate  and  the  moving  average  decrease  with  time.  The  point  estimate  of  the 
failure  rate  for  the  first  year  is  21.9/1000  hours,  for  the  second  year 
17.6/1000  hours,  a  decrease  of  19.6*,  with  an  overall  failure  rate  for  27 
months  of  operation  of  19.2/1000  hours.  The  value  of  chi-square  is  26.6, 
highly  significant.  The  failure  rate  for  this  equipment  is  decreasing  with 
time.  Figure  20  plots  the  average  failure  rate  for  the  14  receivers. 
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Figure  20.  Failure  Rate  Versus  Time  -  Equipment  R-4 
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This  hardware  is  similar  to  R-2,  but  the  results  are  distinctly  dissimilar. 
The  number  of  failures  per  month  and  the  cumulative  failure  rate  versus 
time  are  nearly  constant  (with  the  exception  of  19  failures  in  the  fourth 
quarter  compered  to  an  average  of  33.5  failures  per  quarter,  excluding  that 
quarter).  The  point  estimate  of  the  failure  rate  for  the  first  year  is 
15.0/1000  hours,  for  the  second  year  is  17.9/-100Q  hours,  an  increase  of 
19.3%,  with  an  overall  failure  rate  of  16.9/1Q0Q  hours  for  27  months  of 
operation.  Chi-square  is  11.2,  not  significant  at  the  95%  confidence  level. 
It  is  concluded  that  the  failure  rate  for  this  equipment  is  constant  with 
time.  Figure  21  plots  the  average  failure  rate  for  the  11  receiver  equip¬ 
ments. 
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Figure  21.  Failure  Rate  Versus  Time  -  Equipment  R-5 


This  nardware  is  similar  to  R-3.  The  point  estimates  of  the  failure  rate 
per  quarter  appear  to  oscillate,  but  the  cumulative  failure  rate  versus  time 
is  nearly  constant.  Chi-square  is  0.1,  not  significant.  The  total  plot  is 
based  on  9  failures  the  first  year,  12  failures  the  second  year  and  two 
failures  the  last  quarter.  Hence,  the  individual  points  will  appear  to  vary 
considerably,  but  this  variation  is  due  to  the.  addition  or  subtraction  of 
one  failure.  The  point  estimate  of  the  failure  rate  for  the  first  year  is 
1.1 /10C0  hours,  for  the  seeond  year  1.4/1000  hours,  with  an  overall  failure 
rate  of  1.2/1000  hours.  It  is  judged  that  the  failure  rate  for  this  equipment 
is  constant  with  time.  Figure  22  plots  the  average  failure  rate  versus  time 
for  the  10  equipments. 
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Figure  22.  Failure  Rate  Versus  Time  -  Equipment  R-6 


For  this  equipment,  a  total  of  eight  failures  in  19,580  calendar  hours  were 
experienced,  with  no  more  than  two  failures  recorded  per  quarter.  Failures 
were  observed  as  follows:  three  failures  the  first  year,  four  failures  the 
second  year  and  two  failures  the  last  quarter.  The  chi-square  test  was  not 
performed,  since  the  failure  expectancy  per  cell,  when  grouped,  is  less 
than  five.  The  failure  rate  for  this  equipment  is  increasing,  but  the 
trend  would  be  reversed  if  no  failures  or  one  failure  were  observed  the 
next  quarter.  Based  on  this  data,  it  is  judged  that  the  failure  rate  for 
the  equipment  is  constant  with  time.  Figure  23  shows  the  average  results 
for  the  six  equipments. 
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Figure  23.  Failure  Rate  Versus  Time  -  Equipment  R-7 


Receiver  Equipment  R-8 

This  hardware  is  similar  to  receiver  component  R-7.  A  total  of  14 
failures  in  19,530  calendar  hours  of  operation  were  experienced.  Failures 
were  observed  as  follows,  five  failures  during  the  first  year,  nine 
failures  during  the  second  year,  and  no  failures  during  the  last  quarter. 

Of  the  total  number  of  failures,  five  of  these  were  recorded  during  the 
eighth  quarter  (but  none  in  the  ninth  quarter).  The  chi-square  test  was  not 
performed  since  the  failure  expectation  per  cell,  when  grouped,  is  less  than 
five.  The  cumulative  failure  rate  is  increasing  with  time.  It  is  judged 
that  the  failure  rate  for  this  equipment  is  increasing  with  time,  but  that 
the  trend  could  be  reversed  if  subsequent  operating  quarters  showed  zero 
or  one  failure.  Figure  24  plots  the  average  failure  rate  for  the  four 

equipments. 
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Figure  24.  Failure  Rate  Versus  Time  -  Equipment  R-8 


179 


The  point  estimates  of  the  failure  rate  per  quarter  and  the  cumulative 
failure  rate  versus  time  Increase  with  time,  The  value  of  chi-square  is 
44.7,  highly  significant.  The  point  estimate  of  the  failure  rate  for  the 
first  year  is  4.8/1000  hours,  for  the  second  year  4.8/1000  hours  with  an 
overall  failure  rate  of  5,1/1000  hours  for  ttl2  27  months  @f  operation.  Even 
though  the  failure  rate  por  year  is  constant,  the  failure  rate  per  quarter 
is  not,  a1'  it  Is  judged  that  the  failure  rate  of  this  equipment  is  increas¬ 
ing  with  time.  Figure  25  plots  the  average  failure  rate  of  the  14  equipments. 
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Figure  25.  Failure  Rate  Versus  Time  -  Equipment  C-l 
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Computer  Equipment  C-2 

The  cumulative  failure  rate  versus  time  for  this  equipment  appear  to  be 
constant  with  time.  The  value  of  chi-square  is  9.0,  which  is  not  signifi¬ 
cant  at  the  95%  confidence  level.  The  failurr  rate  for  the  first  year  is 
15.1/1000  hours,  for  the  second  year  is  15.6/1000  hours,  an  increase  of 
3.3%,  and  the  overall  failure  rate  for  27  months  of  operation  is  15.2/1000 
hours.  It  is  judged  that  the  failure  rate  for  this  equipment  is  constant 
with  time  Figure  26  plots  the  average  failure  rate  versus  time  for  the  two 
computers . 
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Figure  26.  Failure  Rate  Versus  Time  -  Equipment  C-2 
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Concur  Equipment  C-3 

This  equipment  exhibited  an  oscillatory  failure  rate  per  quarter,  giving  rise 
to  the  peculiar  cumulative  function.  The  failure  rate  for  the  first  year  Is 
11.1/1000  hours,  for  the  second  year  is  8.1/1000  hours,  and  for  the  total 
27  months  of  operation  is  8.6/1000  hours.  The  only  realistic  conclusion  Is 
that  the  failure  rate  for  this  equipment  is  not  constant.  Figure  27  shows 
the  average  failure  rate  for  the  two  equipments. 
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Figure  27.  Failure  Rate  Versus  Time  -  Equipment  C-3 
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Computer  Equipment  C-4 

For  this  equipment,  a  total  of  13  failures  in  19,530  calendar  hours  of 
operation  were  observed.  The  failures  were  divided  as  follows;  nine  the 
first  year,  four  the  second  year  and  none  the  last  quarter.  The  chi-square 
test  was  not  performed  since  the  failure  expectancy  per  cell,  when  grouped, 
is  less  than  five.  Based  on  the  observed  cumulative  failure  versus  time 
curye,  it  is  judged  that  the  failure  rate  of  this  equipment  is  decreasing 
with  time.  Figure  28  plots  the  average  failure  rate  versus  time  for  the 
two  equipments. 
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Figure  28.  Failure  Rate  Versus  Time  -  Equipment  C-4 
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Computer  Egulggent  C-5 

There  were  a  total  of  21  failures  measured  on  this  equipment,  10  the  first 
six  months  and  11  the  last  18  months  (none  were  recorded  during  the  third 
quarter).  During  the  last  six  quarters  the  failures  were  recorded  as 
follows;  one  failure  in  each  of  two  quarters,  two  failures  in  each  of  three 
quarters  and  three  failures  in  one  quarter.  After  the  first  six  months  of 
operation  the  failure  rate  appears  to  be  constant.  The  value  of  chi-square 
of  2.0  (not  significant)  supports  this  judgement.  Hence,  it  is  judged  that 
the  failure  rate  for  this  equipment  is  constant  with  time.  Figure  29  shows 
the  average  failure  rate  versus  time  for  the  three  equipments. 
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Figure  29.  Failure  Rate  Versus  Time  -  Equipment  C-5 
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This  hardware  is  similar  to  equipment  C-l.  The  point  estimates  of  the 
failure  rate  and  the  cumulative  failure  rate  are  decreasing  with  time  (with 
the  exception  of  the  last  quarter).  The  point  estimate  of  the  failure  rate 
for  the  first  year  is  5.7/1000  hours,  for  the  second  year  is  2.7/1000  hours, 
a  decrease  of  52.7%,  with  an  overall  failure  rate  for  27  months  of  operation 
cf  4.4/1000  hours.  This  failure  rate  was  influenced  by  the  data  from  the 
last  quarter  (14  failures  compared  to  a  total  of  17  the  previous  three 
quarters).  Chi-square  was  26.1,  highly  significant.  It  is  judged  that  the 
failure  rate  for  this  equipment  is  decreasing  with  time.  Figure  30  plots  the 
average  failure  rate  for  the  11  equipments. 
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Figure  30.  Failure  Rate  Versus  Time  -  Equipment  C-6 


Computer  Equipment  C-7 

This  hardware  is  similar  to  equipment  C-2.  The  cumulative  failure  rate 
appears  to  be  constant  with  time.  The  value  of  chi-square  is  10.7,  which 
is  not  significant  at  the  95%  confidence  level.  Tne  failure  rate  for  the 
first  year  is  10.4/10C0  hours,  for  the  second  year  is  10.1/1000  hours,  a  . 
reduction  of  2.9%,  with  an  overall  failure  rate  for  27  months  of  operation 
of  10.9/1000  hours.  This  increase  Is  due  to  32  failures  the  last  quarter. 

It  is  judged  that  the  failure  rate  for  this  equipment  is  constant  with  time. 
Figure  31  plots  the  average  failure  /ate  versus  time  for  the  two  computers. 
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Computer  Equipment  C-b 

This  hardware  is  similar  to  C-3  but  does  not  exhibit  the  oscillatory 
characteristics  of  that  equipment.  The  point  estimate  of  the  failureVate 
and  the  cumulative  failure  rate  are  decreasing  with  time.  The  value  for 
chi-square  is  32.0,  highly  significant.  The  point  estimate  of  the  failure 
rate  for  the  first  year  is  3.0/1000,  for  the  second  year  is  0.35/1000,  a 
decrease  of  88.3%,  with  a  cumulative  failure  rate  for  27  months  of  operation 
of  1,5/1000  hours.  It  is  judged  that  the  failure  rate  for  this  equipment 
is  decreasing  as  a  function  of  time.  Figure  32  plots  the  average  failure 
rate  for  the  two  equipments. 
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Rate  Per  1000  Hours 


This  hardware  Is  similar  to  equipment  C-4.  A  total  of  four  failures  were 
observed,  two  during  the  first  year,,  one  during  the  s<.  ond  year  and  one 
during  the  last  quarter.  The  chi-square  test  is  not  appropriate  due  to  the 
low  failure  expectation  per  cell.  The  failure  during  the  first  year  is 
0.23/1000  hours  and  for  the  last  four  quarters  <s  0.23/1000  hours.  In  six 
of  the  nine  quarters,  zero  failures  were  observed.  It  Is  judged  that  the 
failure  rate  for  the  equipment  is  constant  with  time.  Figure  33  plots  the 
average  failure  rate  for  the  twc  equipments. 


Computer  Equipment  C-10 

i 

This  hardware  is  similar  to  equipment  C-5.  A  total  of  43  failures  were 
observed,  26  in  the  second  quarter  and  11  in  the  fifth  quarter.  In  the 
remaining  six  quarters,  zero,  one  and  two  failures  were  observed  in  each  of 
two  quarters.  Based  on  the  two  high  failure  quarters,  chi-square  was  signi¬ 
ficant  (54.9).  It  is  judged  that  these  two  quarters  represent  anomalies  in 
the  data  reporting  process  and  that  the  appropriate  estimate  of  the  failure 
rate  is  that  it  is  constant  with  time.  Figure  34  shows  the  average  failure 
rate  for  the  two  equipments. 


Quarters 

Figure  34.  Failure  Rate  Versus  Time  -  Equipment  C-10 
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chi-square  test  was  not  performed  since  the  failure  expectancy  per  cell, 
when  grouped,  is  less  than  five.  The  point  estimates  of  the  failure  rate 
appear  to  oscillate,  reaching  a  peak  of  six  failures  in  the  fourth  quarter, 
and  dropping  to  two  failures  in  the  sixth  quarter.  The  only  conclusion 
is  that  the  failure  rate  is  not  constant.  Figure  35  shows  the  average 
failure  rate  for  the  equipment. 


Quarters 

Figure  35.  Failure  Rate  Versus  Time  -  Equipment  C-ll 
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Computer  Equipment  C-13 

A  total  of  nine  failures  were  observed  in  11,912  hours  of  operation.  During 
the  first  two  quarters  one  and  zero  failures  were  observed,  respectively. 
During  tne  next  four  quarters,  four,  three,  one* and  zero  failures  were 
observed,  respectively.  After  the  first  six  months  the  failure  rate  per 
quarter  Is  declining.  The  cumulative  failure  rate  curve  increases  from  the 
first  to  fourth  quarter  and  then  begins  to  decrease.  Based  on  the  point 
estimates  of  the  failure  rate,  it  is  judged  that  the  failure  rate  is 
decreasing  with  time.  Figure  37  plots  the  failure  rate  versus  time  for  this 
equipment. 
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Figure  37.  Failure  Rate  Versus  Time  -  Equipment  C— 13 
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Computer  Equipment  C-H 

Ine  point  estimates  of  the  failure  rate  and  the  cumulative  failure  rate 
yersus  time  are  both  decreasing.  The  value  cf  chi-squa»*e  is  9.2,  which  is 
not  significant.  The  failure  rate  for  the  first  three  quarters  is  3.7/1000 
hours  and  for  the  last  three  quarters  is  1.6/1000  hours,  a  decrease  of  56.71, 
with  an  overall  failure  rate  of  2.5/1000  hours.  The  value  of  chi -square  was 
not  significant  at  the  951  significance  level,  but  it  is  significant  at  the 
901  level  (chi-square  at  901  f  4  degrees  of  freedom  *  7.78).  Based  upon  the 
general  shape  cf  the  curve  and  the  trend  shown  by  the  one  year  moving  average, 
it  is  judged  that  the  failure  rate  of  the  equipment  is  decreasing  with  time. 
Figure  38  shows  the  failure  rate  for  the  equipment. 
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Figure  38.  Failure  Rate  Versus  -  Equipment  C-14 


Computer  Saulpnent  C-15  (Electrical) 

The  nngnetic  tape  unit  failures  were  divided  into  electrical  component  and 
■eehcnical  component  failures.  Figure  39  plots  the  electrical  component 
failures  for  the  equipment.  The  cuuulative  failure  rate  versus  tine  plot 
shout  the  failure  rate  to  be  increasing.  The  failure  rate  for  the  first 
tLree  quarters  is  6.9/1000  hours  and  for  the  last  three  quarters  is  9.7/1000 
hours,  an  increase  of  40.51.  This  increase  is  due  to  the  20  failures 
reported  in  the  fourth  quarter.  The  value  of  chi-square  is  8.0.  idridi  is 
not  significant  at  the  951  confidence  level.  It  is  judged  that  the  failure 
rate  for  the  electrical  conpooeats  of  the  equipoent  is  constant  with  tine. 
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Figure  39.  Failure  Rate  Versus  Tine  -  Equipment  C-15  (Electrical) 
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Figure  40  plots  the  failure  rate  versus  tine  for  the  nedianical  component s  of 
the  Magnetic  tapes.  The  point  estimates  of  the  failure  rates  are  increasing 
with  tine,  as  is  the  emulative  failure  rate  versus  tine.  The  point 
estinate  of  the  failure  rate  for  the  first  three  quarters  is  6.4/1000  hours r 
and  the  failure  rate  for  the  last  three  quarters  is  15.8/1000  hours,  an 
increase  of  147X.  The  value  of  chi-square  is  40.9,  highly  significant.  It 
is  judged  that  the  failure  rate  of  this  equipnent  is  increasing  with  tine. 
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Figure  40.  Failure  Rate  Yersus  Tine  -  Equipment  C-15  (Mechanical) 
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The  point  Ktkites  of  the  failure  rates  per  quarter  are  reported  as  either  ' 
high  (15  failures  in  the  fourth  quarter),  acdiuu  (6-8  failures  in  the  first* 
stead  ani  ninth  quarters)  and  low  (0-2  failures  in  the  third*  fifth*  sixth* 
seventh  ani  eighth  quarters).  The  result  is  a  decreasing  emulative  failure 
rate  vs.  ties.  Chi  -square  is  15. 5,  highly  significant.  The  point  estimate 
of  the  failure  rate  for  the  first  year  is  3.2/1000,  for  the  second  year  is 
0.5/1000,  a  decrease  of  84.4f,  with  an  overall  failure  rate  of  2.0/1000. 

It  is  judged  the  failure  rate  versus  time  is  decreasing  with  tine.  Figure  41 
plats  the  average  failure  rate  for  the  six  consoles. 
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Figure  41.  Failure  Rate  Versus  Tine  -  Equipment  D-l 
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Display  Equipment  D-2 

The  failure  rate  per  quarter  and  the  emulative  failure  rate  are  decreasing 
with  tiwe.  Of  the  total  of  55  tailures  observed,  46  were  recorded  in  the 
first  year.  The  failure  rate  for  the  first  year  is  5.3/1000,  for  the 
second  year  is  0.5/1000,  a  decrease  of  915,  with  an  overall  failure  rate 
for  27  aonths  of  operation  of  2.8/1000.  The  value  of  chi-square  is  47.0, 
highly  significant.  It  is  judged  that  the  failure  rate  for  this  equipment 
is  decreasing  with  tiae.  Figure  42  plots  the  average  failure  rate  for  the 
four  equipments . 
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Figure  42.  Failure  Rate  Versus  Time  -  Equipment  D-2 
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A  total  of  eight  quarters  of  data  were  accumulated  on  this  equipment. 

During  this  time  (17,580  calendar  hours)  a  total  of  5  failures  were  recorded, 
four  the  first  year  and  one  the  second  year.  The  failure  rate  estimate  for 
♦he  first  year  is  0.46/1000  hours,  and  0.12/100Q  hours  for  the  second  year, 
a  decrease  of  73.555.  The  chi-square  test  was  not  performed  since  the 
failure  expectancy  per  cell,  when  grouped,  is  less  than  five.  Based  on 
this  small  matter  of  failures  no  conclusive  judgement  car.  be  made,  but  it 
appears  the  failure  rate  is  decreasing  with  time.  However,  failures  in 
subsequent  quarters  would  reverse  this  trend.  Figure  43  plots  the  failure 
rate  versus  time  for  the  three  equipments. 
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Figure  43.  Failure  Rate  Versus  Time  -  Equipment  D-3 
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This  hardware  is  similar  to  equipment  D-l.  However,  the  results  are  not 
similar.  The  failure  rate  estimates  per  quarter  vary  fro*  zero  failures 
(second  quarter)  to  six  failures  (fourth  quarter),  to  one  failure  (eighth 
quarter)  to  10  failures  (ninth  quarter).  The  cumulative  failure  rate  curve 
rises,  then  dips  and  finally  rises  again,  (the  curve  shown  in  Figure  44 
smooths  these  variations).  The  value  of  chi-square  is  6.3,  which  is  signi¬ 
ficant  at  the  951  confidence  level.  Based  on  the  variation  of  the  failure 
rate  estimates  per  quarter,  the  only  statement  that  is  realistic  is  that 
the  failure  rate  is  not  constant.  Figure  44  plots  the  average  failure  rate 
versus  time  for  the  four  equipments. 
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Figure  44.  Failure  Rate  Versus  Time  -  Equipment  D-4 
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This  hardware  is  sinilar  to  equipment  0-2.  Tne  failure  rate  point  estimates 
and  the  cumulative  failure  rate  versus  time  are  decreasing.  The  point 
estimate  of  the  failure  rate  at  the  end  of  one  year  is  6.1/1000  hours,  for 
the  second  year  is  2.3/1000  hours,  a  decrease  of  62.1*,  with  an  overall 
failure  rate  for  27  norths  of  operation  of  3.8/1000  hours.  The  value  of 
dii-square  is  48.0,  highly  significant.  It  is  judged  that  the  failure  rate 
for  this  equipment  is  decreasing  with  tine.  Figure  45  plots  the  average 
failure  rate  versus  tine  for  the  three  equipments. 
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Figure  45.  Failure  Rate  Versus  Time  -  Equipment  D-5 
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A  total  of  *0,850  calendar  hours  of  operation  were  recorded  and  one  failure 
was  observed.  This  failure  occurred  during  the  first  quarter  of  reporting 
on  the  equipment.  Hence,  the  cumulative  failure  rate  versus  time  is 
decreasing.  The  chi-square  test  was  not  performed  as  insufficient  data  is 
available.  Based  on  this  snail  quantity  of  data,  it  is  not  reasonable  to 
judge  whether  the  failure  rate  is  constant  or  decreasing.  Figure  46  plots 
the  average  failure  rate  versus  tine  for  the  three  equipments. 
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Figure  46.  Failure  Rate  Versus  Time  -  Equipment  D-6 
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The  point  estimates  of  the  failure  rate  per  quarter  and  the  cumulative 
failure  rate  versus  time  are  both  constant.  The  value  of  chi-square  is 
3.1,  which  is  not  significant  at  the  95X  significance  level.  The  failure 
rate  estimate  for  the  first  three  quarters  is  3.0/1000  hours,  and  for  the 
last  three  quarters  is  3.1/1000  hours,  an  increase  of  3.3X.  It  is  con¬ 
cluded  that  tne  failure  rate  of  this  equipment  is  constant  with  time. 

Figure  47  plots  the  failure  rate  versus  time  for  the  eleven  display  consoles. 
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Figure  47.  Failure  Rate  Versus  Time  -  Equipment  D-7 
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This  equipment  is  a  camera  display  unit  for  a  large  dynamic  display.  A 
total  of  28  failures  being  recorded.  The  failure  rate  per  quarter  is 
26/1000  hours,  21.3/1000  hours,  10.2/1000  hours  and  4.7/1000  hours.  The 
value  of  chi-square  is  9.7,  which  is  significant.  Although  less  than  2000 
hours  of  actual  operation  were  accumulated,  it  is  judged  that  the  failure 
rate  for  this  equipment  is  decreasing  with  time.  Figure  48  plots  the  failure 
rate  versus  time  for  this  display  equipment. 
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Figure  48.  Failure  Rate  Versus  Time  -  Equipment  D-8 
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This  equipment  accumulated  a  total  cf  nine  failures  in  3164  hours  of  opera¬ 
tion.  The  failure  rate  per  quarter  is  8.1/1000  hours,  4.4/1000  hours, 
3.6/1000  hours,  and  0.6/1000  hours.  The  failure  rate  trend,  both  for  the 
point  estimates  per  quarter  and  the  cumulative  failure  rate  versus  tine 
are  decreasing.  Chi-square  was  not  computed  since  the  failure  expectancy 
per  cell  did  not  exceed  five.  Although  only  3164  hours  of  operation  were 
accumulated,  it  is  judged  that  the  failure  rate  for  this  equipment  is 
decreasing  with  time.  Figure  49  plots  the  failure  rate  versus  time  for  this 
equipment. 
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Figure  49.  Failure  Rata  Versus  Time  -  Equipment  D-9 
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failure  Kate  ts  Time  Characteristic  of  the  Equipment 


This  section  presents  the  results  of  the  examination  of  the  failure 
rate  7S  tine  characteristics  of  the  equipments  analyzed  in  the  study. 
Table  LXXY  presents  a  sunary  of  the  deta  presented  in  6.C.  (2)  above. 
This  table  thews  that  27.1  percent  of  the  equipnemts  have  failure 
rate  vs  tine  characteristics  tfeich  could  be  considered  constant  with 
tine,  and  72.9  percent  of  the  equipment  have  failure  rate  vs  tine 
characteristics  which  are  judged  not  constant  with  tine.  Based  on 
the  total  equipment  count,  45.9  percent  have  failure  rate  vs  tine 
characteristics  idiich  are  decreasing  with  tiwe,  10.8  potent  have 
failure  rate  vs  time  characteristics  that  are  increasing  with  tine, 
and  16.2  percent  have  failure  rate  vs  tin e  characteristics  which  are 
oscillating  with  tiwe  or  for  tdrich  sufficient  data  does  not  exist  to 
estiwate  the  change  in  fails-*  rate  vs  tine.  These  determinations 
(increasing,  decreasing,  constant  or  not  determinable  from  the  data) 
are  based  upon  the  chi-square  test  for  goodness  of  fit  for  statistical 
verification,  and  on  eng i leering  judgnent. 

Table  LXXY!  shows  the  change  in  failure  rate,  coopering  the  point 
estimate  of  the  failure  rate  for  the  first  year  with  the  point  esti¬ 
mate  of  the  failure  rate  for  the  second  year.1  If  the  failure  rate 
for  the  first  year  exceeded  the  failure  rate  for  the  second  year,  the 
following  ratio  was  established: 


i  Second  Year  -  1  First  Tear 

i  Wend  Year  x  M 


-« 


This  value  (-KX)  is  the  percentage  failure  rate  decrease  from  the 
first  year  to  the  second  year.  For  example,  if  x  first  year  *  10/1000 
hours  and  x  second  year  *  5/1000  hours. 


-X  =  x  10G  *  -1005 


This  Beans  a  two  to  one  decrease  in  the  failure  rate  was  observea  from 
the  first  year  to  the  second  year. 


1.  In  the  situation  where  wily  six  quarters  of  data  were  available,  compari¬ 
sons  were  cade  between  the  first  three  quarters  end  the  last  three  quarters. 
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If  the  failure  rate  for  the  second  year  exceeded  the  failure  rate  for 
the  first  year,  this  second  ratio  wts  established. 


>_  Second  Tear  -  x  First  Tear  _ 

- msrfar - 1  100 


♦X5 


For  example,  i*  x  first  year  *  5/1000  hours  and  x  second  year  *  10/ 
1000  hours. 


K 


x  100 


♦l  005 


This  Mans  a  two  to  one  increase  in  the  failure  rate  was  observed  frog 
the  first  year  to  the  second  year. 

Table  LXXV  presents  these  percentages  for  those  equipments  which  hive 
been  judged  to  have  constant,  decreasing  and  increasing  failure  rate 
v$  tine  characteristics.  All  equipment  included  in  Table  LAX?  is 
included  In  Table  LXXYI  except  two  display  equipments  (0-8  and  0-9) 
litere  year  t©  year  comparisons  were  not  possible  as  only  one  year  of 
data  was  collected  on  those  two  equipments. 

Table  LXXYI  shows  that  48.45  of  the  equipments  {14  equipments)  experi¬ 
enced  failure  rate  decreases  greater  than  505, and  20.75  of  the  equip¬ 
ments  (6)  experienced  failure  rate  decreases  between  zero  and  505. 

A  total  of  30.95  of  the  equipments  experienced  failure  rite  increases 
and  of  those.  2D. 75  experienced  failure  rate  increases  of  up  to  505. 

In  three  cases  the  failure  rate  increase  was  greater  than  5GS.  For 
those  equipments  where  the  failure  rate  is  judged  to  be  constant  with 
time,  905  of  the  equipments  experienced  failure  changes  of  plus  or 
minus  505.  The  one  equipment  which  did  not  fall  into  this  range 
(C-10),  experienced  a  large  number  of  failures  during  the  second  re¬ 
porting  period  wnich  led  to  a  decreasing  failure  rate  vs  time  charac¬ 
teristic.  However,  this  point  was  judged  to  be  an  anomaly  in  the  data 
and  the  failure  rate,  ba^ed  or.  the  remaining  data,  was  judged  to  be 
constant  with  time.  Further,  of  the  9  equipments  whose  failure  rates 
are  judged  constant  (chi-square  test  and  engineering  judgment),  six 
equipments  experienced  increases  in  failure  rates  from  the  first  to 
second  year  of  operation.  Hence,  it  appears  that  an  increase  in  fail¬ 
ure  rate  can  be  observed  (67?  of  the  cases)  and  tee  constant  failure 
rate  assumption  is  not  invalidated. 
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TABLE  LJXY  EQUIP**!  FAILURE  RATE  CHARACTERISTICS  VS  TIME 


Equipment  Constant 
Type  Failure  Rate 


Receiver  I  3 


Decreasing 
Failure  Rate 


■9 


9 


Increasing  lUraJetenrined 


CoKouter 


6 


5 


2 


3 


SECTION  IX 
RECG*€KQATIOKS 


Investigation  should  be  continued  of  the  relationships  established  in 
this  study  between  prediction  accuracy  and  the  following  equipment 
characteristics: 

o  Design  aporoach  category 

o  Functional  category 

o  Hon- Part  related  failure 

o  Decreasing  equipment  failure  rates 

o  Part  Technologies 

Investigation  should  be  continued  of  the  relationship  established 
between  prog  raw  reliability  ratings  and  prediction  accuracy. 

A  formal  prediction  by  similarity  technique  should  be  developed  to 
include  a  oetailea  data  baseline  representing  a  complete  range  of 
electronic  circuits,  sub- assemblies  and  equipment  for  a  full  range 
of  ground  environments. 

Specific  elements  of  the  RADC  Notebook,  Volune  II  stress  analysis 
prediction  technique  which  the  study  indicates  to  require  revision 
and  further  development  include: 

o  Part  quality  grade  modifiers 

o  Semi-conductor  failure  rate  modifiers  for  digital  and  analog 
equipment. 
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APPENDIX  I 


PROGRAM: 

LANGUAGE: 

MACHINE: 

PURPOSE: 


ASSUMPTIONS: 


MULTIPLE  LINEAR  REGRESSION  PROGRAM 


MLR-1 

FORTRAN  IV 
IBM  7094 

This  program  uses  the  least  square  technique  to  estimate  the 
constant  (a)  and  the  coefficients  (bj)  in  the  equation; 

=  3  +  b-j  K-j  +  b2  FU  +  — -  +  bN  Kn 
o 

In  addition,  the  program  generates  statistical  quantities  such 
as 

Multiple  R 

Standard  error  of  estimate 
Analysis  of  variance  table 
Covariance  and  correlation  matrices 
The  least  squares  method  is  a  valid  technique  to  obtain  the 
best  linear  fit  to  a  set  of  data  regardless  of  the  nature  of 
data.  However,  in  order  to  use  the  statistical  quantities 
mentioned  above  the  following  assumptions  must  be  made. 

1.  The  dependent  variable  *pAQ  can  be  expressed  as  follows: 

V xo  =  a  +  bi  Ki  b2  K2  +  '***  +  bN  +  £ 
where 

c  is  a  normally  distributed  random  variable  with  mean  =  0. 

2.  For  any  set  {K^l  the  variance  of  Xp/XQ  is  the  same. 

3.  The  error  of  the  estimate  i..  any  observation  is  uncorrel -»ted 
with  the  error  in  anv  other  observation. 


WUT:  Inputs  required  to  run  a  problem  Include: 

1.  Staple  size 

2.  Huaber  of  variables 

3.  Transgeneration  cards  (used  to  transfer*  input  variables 
and  create  new  variables) 

4.  Saaple  values 

5.  F  level  for  inclusion 

6.  F  level  for  exclusion 

7.  Tolerance  level 

OUTPUT:  Outputs  froa  the  prograa  include: 

1.  Multiple  R 

2.  Standard  error  of  estisate 

3.  Analysis  of  variance  table 

4.  For  each  independent  variable 

a.  Regression  coefficient 

b.  Standard  error 

5.  Means  and  standard  deviation 

6.  Covariance  and  correlation  matrices 

7.  List  cf  residuals 


1-2 


A??£3©il  II 

SITES:  ASH.YSIS  fjedi  trios  ?2DS£B5 


paosaw  ?as  i 

USgtfg  HKT&X  l¥ 
JfcOillE  2  Z3S 


££SC?a?Ti£M 

This  program  is  csss  to  oesarate  tie  prediction  variability  crrves  described 


in  tie  text  of  tie  final  report.  Tie  equation  =*sed  to  ccmpute  system  failure 
rats  at  a  fixed  environment,  -art  grade,  stress  and  temperature  level  is: 


;-s 


x 


liiere  JS 
% 


1 


"£i  1  - 
•  >  * 

S'J.k 


The  number  of  different  types  of  parts  in  tie  system. 

The-  number  of  parts  cf  type  i. 

Failure  rate  cf  ,»rt  3c  at  stress,  tenperature  level  j 
(as  determined  by  Volume  II). 

The  product  of  the  non-envi rosmental  x  factors  for  part  k. 

Part  Quality  grade  indicator,  i.e.,  1  =  upper 

2  =  lower 

Environment  indicator,  i.e.,  1  --  spacecraft 

2  -  ground 

3  «  rsissile 

The  Yolume  II  environmental  factor  for  part  k  of  relia¬ 
bility  grade  i  used  in  environment  1. 

The  Volise  II  factor  of  pare  k  of  reliability  grade  i 
when  used  in  environment  1. 


II-l 


imr  , 

Hk  penptatnt  data  to  the  program  assists  of: 

%  for  each  type  of  ca$>csaesi 

(a)  A  ta&Te  of  cowpcoest  failure  rates  -for  pre-detendned  levels 
of  stress  aad  teaperatsre. 

(fe)  The  product  cf  the  coa-eayircaaeatel  t?  factors. 

(c)  »se  esriroErrstsl  t_  factor  aad  the  factor  for  each  of  three 
ONjitnots  *ad  each  of  tuo  part  grades. 

To  cowpcfte  the  failure  rate  of -a  gives  system  operatise  is  a  g>«a  eaviroc- 
seet,  tk  fbHodag  diis  is  rests? red: 

(a)  fe  onri  rawest  iadicaior 

{i}  The  qsetity  of  each  type  of  part 

wim 

Gives  this  isforsatioe  the  prsgran  coHperfcss  a  faarily  of  system  failsre  rates 
vaiyiag: 

{*)  Stress 

(b)  Tejaperatsare 

(c)  Part  grade 

In  addition,  the  prog  ran  prints  the  qumtity  of  each  part  type  along  with  its 
percent  contribution  to  the  overall  system  failure  rate. 

LOAD  SHEETS 

The  Input  to  the  prograa  is  “free  format.*  That  is,  the  user  merely  enters; 
a  series  of  makers  separated  by  cgeses,  where  the  first  rxsbsr  indicates  the 
enviroraent  and  the  resaining  rasher  indicates  the  quantity  of  each  part  type. 
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?Istsl  Report  -  2*  Apr  67  U,  2k  Mty  68 
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Gerald  Pittler 
Robert  L  Saelton,  et  *1 
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7a.  TOTAL  MO. 
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EA3BC  rSCJECY  HKIE22: 
Arthccy  J.  Feduecia  (3S2S) 
AC  315  330-4064 


Bcae  Air  Dewelopaent  Center  (3HZ8B) 
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IS  *t»T«I 


This  stmjy  evaluates  the  accuracy  of  pre-design  and  stress  analysis  reliability 
prediction  techniques,  including  the  HADC  Eel  lability  Xctebcok,  Tolwe  II ,  stress 
analysis  method,  when  applied  to  a  variety  of  ground  electronic  eeuipnent.  Sources 
of  prediction  inaccuracy  are  investigated  and  identified.  Program-related  factors 
significant  to  the  achieveaect  of  reliability  and  prediction  accuracy  are  identified 
«r>a  a  quantitive  rating  system  is  established  and  related  to  system  prediction 
accuracy. 

Sew  and  improved  pre-design  and  stress  analysis  reliability  prediction  aethods 
are  developed  and  tested.  Equipment  design  approach  categories  having  different 
prediction  accuracy  characteristics  are  identified  with  statistical  distributlcns  of 
prediction  accuracy  ratios.  Degradation  analysis  processes  and  techniques  are 
\  identified,  evaluated,  and  presented  with  a  recoomended  approach  for  their  applicable 
?  Reliability  demonstration  aethods,  inclvding  the  Bayesian  approach,  are  evaluated. 

A  reccassended  reliability  demonstration  approach  for  ground  electronic  equipment  is 
developed.  A  new  base  for  integrated  circuit  failure  rates  is  also  provided. 
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